*nix Tools and References

fractalsfractals Regular
edited August 2010 in Tech & Games
I thought it would be a good idea to have a thread with common but less-known (and those with easily forgettable names) utilities and general references. Lots of times I stumble across GNU [or other common] utility that would be really useful, but when I actually need to use it I can't remember what it's called and I end up using all the time I would have saved to find it through google. I'm going to skip cp, ls, mv, etc. I hope you all don't mind.

You should also post here if you need to do something but cant find the tool to do it. Even if no one replies in time to help you, it can be added to the list.

Well-Known Utilities
for teh n00bz
  • man - displays a manual page
  • cat - combines 2 files and prints to stdout
  • grep - searches for a string and prints every line it occurs on (or just the match, or any line it doesn't appear on)

Less-Known Utilities
for when you cant remember the damn name
  • sed - sed - stream editor for filtering and transforming text
  • tee - read from standard input and write to standard output and files
  • less - lets you scroll through a file or stdin
  • tail - print the last 10 (or specified number) lines
  • file2cable - sends a binary file through a network interface, as is

References
I'll keep updating these, especially the less-known ones as I find them, but please post suggestions and I'll add it to the OP.

Comments

  • OnesanOnesan Acolyte
    edited August 2010
    file2cable and tcpreplay, for some reason i keep forgetting file2cable and thinking file2wire handy for "playing" network capture files to a network interface.

    oh and please do something on pipes, pipes are great fun.

    cat /var/log/httpd/access_*|grep somewebsite.com|awk '{print $1}'|sort|uniq > /var/log/shit/whateversitesips.txt
  • fractalsfractals Regular
    edited August 2010
    A stream can be thought of as a 1-way road for data to travel. Another good analogy is a stream(river), where your data would be the water, flowing in only 1 direction. (It's important to remember computers are magic; 2 parallel streams can flow in opposite directions, ignoring the law of nature.) Streams in *nix are inherited from the C programming language, which gives programs 3 standard streams, stdin, stdout, and stderr. In addition to the 3 standard streams, files, network connections, serial connections, and more are handled as streams by *nix OS's, but unless you are a programmer you will only deal with the standard streams.

    stdin is the standard input stream. By default, it's tied to the terminal input, so anything you type is sent to stdin.

    stdout is the standard output stream, which, by default is tied to the terminal. Functions in C like printf() output to stdout.

    stderr is the standard error stream. Programs generally send error messages through this stream. By default it is tied to the terminal output, just like stdout. Having a separate stream for errors can be useful when you redirect stdout.

    You can redirect the standard streams to other programs or to a file by placing a character between the program and the file or program to reassign the streams to. To redirect stdout to a file, use the '>' character.
    [email protected]:~$ echo "redirected stdout" > ./test
    
    That will create a file called 'test' in the current directory, with the words "redirected stdout." If you use '>>' instead, the output will be added on to the end of the file, instead of clearing the file first. If you add a '&' next to the '>' (before or after), it will redirect stderr too. '<' can be used in a similar way to redirect stdin to the file. The pipe character ("|," next to the ] and enter) is used to create a pipeline between two programs. The first program's stdout is redirected to the second programs stdin.
    [email protected]:~$ cat really_long_text_file | less
    
    That will redirect the output from really_long_text_file to 'less,' which allows you to scroll through blocks of text that are longer than a page. It's less useful with terminal emulators (terminal programs, they usually have a scroll bar on the side), but with a true terminal it's more helpful. You could get the same results if you redirected really_long_text_file directly into less using the '<' character, since cat just reads a file (or multiple files) and sends it to stdout.

    Here's a more practical example:
    [email protected]:~$ nmap 192.168.0.0/16 -oG | grep up | grep -e [0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3} | uniq | sort > ./openports
    
    That's written by memory, but it should scan every IP in the local network (every IP between 192.168.0.0 - 192.168.255.255), grep out every line containing an IP assigned to a computer that's up, extract those IP's, then remove the duplicates and sort the list. Finally, it will put that list of active IPs in the file 'openports.'
Sign In or Register to comment.