I thought it would be a good idea to have a thread with common but less-known (and those with easily forgettable names) utilities and general references. Lots of times I stumble across GNU [or other common] utility that would be really useful, but when I actually need to use it I can't remember what it's called and I end up using all the time I would have saved to find it through google. I'm going to skip cp, ls, mv, etc. I hope you all don't mind.
You should also post here if you need to do something but cant find the tool to do it. Even if no one replies in time to help you, it can be added to the list.
Well-Known Utilities
for teh n00bz
- man - displays a manual page
- cat - combines 2 files and prints to stdout
- grep - searches for a string and prints every line it occurs on (or just the match, or any line it doesn't appear on)
Less-Known Utilities
for when you cant remember the damn name
- sed - sed - stream editor for filtering and transforming text
- tee - read from standard input and write to standard output and files
- less - lets you scroll through a file or stdin
- tail - print the last 10 (or specified number) lines
- file2cable - sends a binary file through a network interface, as is
References
I'll keep updating these, especially the less-known ones as I find them, but please post suggestions and I'll add it to the OP.
Comments
oh and please do something on pipes, pipes are great fun.
cat /var/log/httpd/access_*|grep somewebsite.com|awk '{print $1}'|sort|uniq > /var/log/shit/whateversitesips.txt
stdin is the standard input stream. By default, it's tied to the terminal input, so anything you type is sent to stdin.
stdout is the standard output stream, which, by default is tied to the terminal. Functions in C like printf() output to stdout.
stderr is the standard error stream. Programs generally send error messages through this stream. By default it is tied to the terminal output, just like stdout. Having a separate stream for errors can be useful when you redirect stdout.
You can redirect the standard streams to other programs or to a file by placing a character between the program and the file or program to reassign the streams to. To redirect stdout to a file, use the '>' character. That will create a file called 'test' in the current directory, with the words "redirected stdout." If you use '>>' instead, the output will be added on to the end of the file, instead of clearing the file first. If you add a '&' next to the '>' (before or after), it will redirect stderr too. '<' can be used in a similar way to redirect stdin to the file. The pipe character ("|," next to the ] and enter) is used to create a pipeline between two programs. The first program's stdout is redirected to the second programs stdin. That will redirect the output from really_long_text_file to 'less,' which allows you to scroll through blocks of text that are longer than a page. It's less useful with terminal emulators (terminal programs, they usually have a scroll bar on the side), but with a true terminal it's more helpful. You could get the same results if you redirected really_long_text_file directly into less using the '<' character, since cat just reads a file (or multiple files) and sends it to stdout.
Here's a more practical example: That's written by memory, but it should scan every IP in the local network (every IP between 192.168.0.0 - 192.168.255.255), grep out every line containing an IP assigned to a computer that's up, extract those IP's, then remove the duplicates and sort the list. Finally, it will put that list of active IPs in the file 'openports.'