pipes do not pipe

[semenaa]

The script's purpouse is to read input from some device and apply a filter on it.

This pipe doesn't work as it I thought it should:

# netcat 172.17.1.65 33001 | grep "@" | cut -f9 | perl -wnE '$w+=scalar(0);print "$w $_"'

these constructions show output but do not write to files (why?):

# netcat 172.17.1.65 33001 | grep "@"
# netcat 172.17.1.65 33001 | cut -f9
# netcat 172.17.1.65 33001 | perl -wnE '$w+=scalar(0);print "$w $_"'

What's wrong? Hasn't found anything in man/info pages. It seems like I use it somehow wrong.

 

[JasKinasis]

From an initial look, the script looks like it should be OK. I definitely don't think it's a problem with the pipes themselves. The concept of pipes have been around for at least 40 years. So they definitely work, heh heh!

The pipes between netcat, grep and cut are definitely OK, grep and cut can take input from stdin (piped from previous command). Which just leaves the perl part of that line. But you say that your 'constructions' in your previous post all return output, so the pipe to perl must also be OK too.

If you run your script like this:

bash -x /path/to/yourscript.sh

It will run your script in a shell with debugging turned on. The output from running your script like that might yield some clues as to what is going wrong.

Otherwise, try building the command up bit by bit and see where it's failing:

netcat 172.17.1.65 33001
netcat 172.17.1.65 33001 | grep "@"
netcat 172.17.1.65 33001 | grep "@" | cut -f9
netcat 172.17.1.65 33001 | grep "@" | cut -f9 | perl -wnE '$w+=scalar(0);print "$w $_"'

etc....
At each step, ensure that the output you are seeing is what you are expecting to see!

As for why it isn't writing any files: You aren't writing anything to a file, to do this you'd need to redirect the output to a file.
e.g.

netcat 172.17.1.65 33001 | grep "@" >> /path/to/output.txt

The exact redirection to use would depend on what output you want in your file (stdout, stderr, or both) and whether you want to overwrite the file each time the command was ran, or append to the end of the file. Also, if you want to see the output on the screen AND have it redirected to a file, you would need to use the tee command too!

I haven't got time to explain the ins and outs of redirection ATM, but if you do a search for 'linux redirect to file' you should be able to find tons of information on I/O redirection!

 

[semenaa]

Thanks, I have an idea. Probably both grep and cut expect EOF to transmit text into pipe or write it to a file (through redirection, of course). In my case, netcat goes on and on with transmission without EOF, so the decision is a single all-in-one perl script to work on each line or a temporary file.