Using Perl

Discussion in 'Intermediate Tutorials' started by Rob, Jul 9, 2013.

  1. Rob

    Rob Administrator Staff Member

    Joined:
    Oct 27, 2011
    Messages:
    606
    Likes Received:
    270
    Your skills as a system administrator will improve immeasurably if you learn Perl - even just a working knowledge of it.

    What is Perl

    Perl is a general purpose language that can be used for system administration. In fact, it was created by Larry Wall specifically for that purpose. Many of the examples that we have seen before in which we used other separate programs can be done by using Perl alone. This section is not meant to be an exhaustive explanation on how to use Perl nor should it be considered a tutorial on the Perl language. Before using Perl, you should familiarize yourself with it by typing:

    Code:
    perldoc perlintro
    Which will give you a basic tutorial. Perl is installed on almost all standard Linux distributions, so perldoc should be available.

    If you use perl for system administration, you will find yourself using a lot of short perl expressions known among administrators as one-liners. Here are some one-liners that you may want to study, change and adapt and then use later on.

    The following will print out specific lines of a file just like more, less or cat.

    Code:
    perl -ne 'print if 1 .. 5' our_file
    We can also choose not to print lines:

    Code:
    perl -ne 'print unless 1 .. 5' our_file
    The unless option means not to print lines 1 through 5.

    One of the best ways of using perl one-liners is as a substitute for sed Here is an example of how we use Perl like sed:

    Code:
    perl -p -i.bkp -e 's/SCO/Satan/' SCO_news
    Here, we have substituted the word SCO for the word Satan in the file SCO_news and besides, we have created a backup file. This is a good idea because if we hadn't used the -i.bkp option, we would have overwritten the SCO_news file forever.

    We can also use Perl like awk. Here is an example

    Code:
    ls -l | perl -lane 'print $F[+8]' 
    This will print out just the name of the file. That's not particularly useful because we can do this just as easily by typing ls -1 . But if we add a little pizazz to it, it becomes extremely useful.

    Code:
    ls -l|perl -lane 'print join " ",@F[5..8]'
    This will print out the month, year and hour of the file's creation or last modification and the file name.

    Here's another one that you may find useful. If you know someone who's a caps lock freak, they may name all their files in CAPITAL LETTERS. Here's a way to use perl to rename files to lower case names.

    Code:
    perl -e 'for (@ARGV) { rename , lc() ; }' *
    All of the files (*) in that particular directory will be renamed to the lowercase equivalent.

    If you happen to be like me and you feel you don't have to use a fancy word processor with bells and whistles in order to write what you're thinking, this following one-liner may come in handy. The spell checker that I use, ispell, won't catch the occasional upper-case letter that should be lower case. I will usually mess up when it comes to a person's title vs. talking about him/her generically. (eg. President Clinton or Clinton was the president in 1999) This will print all the words that begin with a capital in the file. You can then make corrections if you've capitalized something that shouldn't be.

    Code:
    perl -ne 'push@w,/(b[A-Z]S*?b)/g;END{print"@w"}'
    As far as system administration goes, if you've written some script, you might want to check it for capitalized words. Remember /home/bob/Report and /home/bob/reports are two different directories, so this could come in handy to debug some problem with a script. Typos cause a lot of heartache to programmers as you may know!

    Speaking of heartache (or rather headache) the problem of spam has reached epidemic proportions. It's fairly easy to lock down a mail server, but you never know if people from inside your organization, particularly if you're an administrator of a school, college or university, are giving aid to spammers. Here's a way to get a list of who's on the receiving end of mail coming from your server. We'll get list of addresses to which we've sent from the logs of the Postfix mail server.


    Code:
    perl -ne 'print "$1n" if /sto=<([^>]+)>.*?relay=(?!local,)/' /var/log/mail.log
    Alone, this is not an effective spam fighting tool, but if you see mail addresses with clear patterns, you may be on to some spam activity.

    As you can see, Perl is pretty powerful on its own. We can also use Perl with the other utilities we've seen before to get an extra boost of power.

    Using Perl with other utilities

    As previously mentioned, you can combine Perl with other utilities to increase the manageability of the things you do with it. Here are some examples.

    With the following, you get a list of open ports and who's using them:

    Code:
    cat /proc/net/tcp |perl -lane '(undef,)=split ":",; print hex()."t".getpwuid() if '|sort -n|uniq -c
    This is tremendously important. Ports that aren't needed should be closed. Any open port is a potential source of unwanted entry. Here we have passed a reading of our TCP ports to Perl and then, as we saw earlier, we sort them and filter out any repeat entries with uniq.

    Some of our previous examples were regarding the Apache web server access logs. Well, here's another one for that group. This is a simple one-liner that gives us an ordered count of "hits" on our website from most visited to least.

    Code:
    cat access | perl -ne 'print "$1n" if /GET (.*?) HTTP/'|sort|uniq -c|sort -nr|less
    It's a good idea to dabble a little in Perl, even if you're not planning on using it for web projects and other programming endeavors. You can get more information at the perl site: http://www.perl.org/.
     
  2. omoresec

    omoresec New Member

    Joined:
    Nov 7, 2013
    Messages:
    2
    Likes Received:
    2
    maybe you could post python
     
    DevynCJohnson likes this.
  3. DevynCJohnson

    DevynCJohnson Well-Known Member Staff Member Staff Writer

    Joined:
    May 21, 2013
    Messages:
    1,635
    Likes Received:
    1,273
    I recently submitted a Python3 tutorial. It will be released for the public in about a week. Enjoy!
     
  4. Saul Briones

    Saul Briones New Member

    Joined:
    Mar 4, 2014
    Messages:
    5
    Likes Received:
    4
    In your opinion, between Perl and Python, which one of the two is the best for system administration and why? Thank you!
     
    DevynCJohnson likes this.
  5. DevynCJohnson

    DevynCJohnson Well-Known Member Staff Member Staff Writer

    Joined:
    May 21, 2013
    Messages:
    1,635
    Likes Received:
    1,273
    I would suggest Python because it is a language that can perform many tasks while Perl specializes in text manipulation (like awk and sed).
     
  6. rkromer

    rkromer New Member

    Joined:
    Oct 2, 2015
    Messages:
    6
    Likes Received:
    3
    Hmm... may I disagree? For system administration I suggest Perl, because you can use it any way you wish:
    • as Perl script,
    • as traditional, procedural Perl programme and
    • as full object-oriented Perl programme.
    In Python it's more tedious to achieve a quick "shell"-like task.
     
    Last edited: Oct 2, 2015
    Jon Brookes and DevynCJohnson like this.
  7. Nathan Rodriguez

    Nathan Rodriguez New Member

    Joined:
    Oct 1, 2015
    Messages:
    14
    Likes Received:
    9
    Think both are powerful enough to manage system administration tasks, even "simple" shell scripting has an enormous power for the versatile coder, ending in a matter of what tool is more familiar or the need of a specific library.
     
    DevynCJohnson likes this.
  8. Jon Brookes

    Jon Brookes New Member

    Joined:
    Oct 10, 2015
    Messages:
    3
    Likes Received:
    4
    Having been in systems admin since 1998 I have used Perl to script for Windows and Unix, various flavours of each but mostly Linux.

    Perl is used extensively in many areas of systems and application programming, for example Nagios plugins are for the most part written in Perl. Many plugins for security software likewise are written in Perl. It can be found in so many places you at least need to be able to read and make sense of Perl code. Even Puppet was at one time developed in Perl until the then author chose to take a different route and started over in Ruby.

    Whenever I have a system daemon to write, a web service, API, mail report, whatever, Perl and specifically CPAN are the first ports of call for me at the very least to get a working prototype running. Often as not, what you you can't do in Perl is not worth worrying about.

    Whilst Python was around when I first started coding in Perl, Perl was at the time the language of choice for both systems and web developers alike. The 'Camel' books were on the shelves of most programmers I came across. Over time however, for one reason or another, PHP being one of them, Perl has lost the popularity it had in times past. Conversely now many starting out in coding may have never heard of or considered using Perl. This is due to a number of reasons, Python is taught in schools and PHP has become the web server language of choice. Wordpress for instance, being written in PHP and which borrows some of Perl's syntax, has long since overtaken the former top spot that Moveable Type had (which was partly written in Perl) and which fell out of favour over licensing.

    If you want to learn more about Perl and it's everyday use, Perl Maven is a recent, up to date source of learning and advise. There are many answers to common and not so common questions and problems at Perl Monks. CPAN is of course a one stop for pretty much any problem you can think of. It is also where you will find some of the best documentation complete with example usage anywhere. When looking for something similar with other languages I have often gone back to Perl, as the level of quality documentation and tested code seems lacking in comparison.

    Looking forward, the one 'scripting' language that seems to be one to watch is Javascript. Like it or no, it has the potential to run on more devices, thanks to the prevalence of mobiles, tablets and the success of node.js than any other so called 'scripting' language.

    Whichever language you choose to write in however, don't just learn one. Whilst you used to be able to see that 'one language ruled them all', certainly in terms of scripting, this cannot be said any more due to the multiple rise of new languages and frameworks. Likely look at Perl, Python, Ruby, watch out for Javascript and if your really serious about systems programming and want to try to avoid the verbosity of C and C++ together with memory management issues, take a look at Go which borrows from Python in some of it's syntax and tries to be at home with those accustomed to 'scripting languages'.
     
    Last edited: Oct 10, 2015
    rkromer and DevynCJohnson like this.
  9. DevynCJohnson

    DevynCJohnson Well-Known Member Staff Member Staff Writer

    Joined:
    May 21, 2013
    Messages:
    1,635
    Likes Received:
    1,273
    @Jon Brookes , thanks for the info. You changed my opinion about Perl. I used to think that Perl was not very useful for a variety of tasks besides text manipulation.
     
  10. dtse9

    dtse9 Member

    Joined:
    Jun 19, 2015
    Messages:
    80
    Likes Received:
    10
    why not c or vdhl (Very-High-Speed Integrated Circuit Hardware Description Language)?
    does the os of phone use perl or python?
    usually, the memory of the hardware less than 256 mib, sometimes less than 20mib. what can you do?
    usually , we have to only use the linux kernel.
     
    Last edited: Feb 21, 2016
  11. dtse9

    dtse9 Member

    Joined:
    Jun 19, 2015
    Messages:
    80
    Likes Received:
    10
    thanks a lot. i had find it for a week.
     
  12. dtse9

    dtse9 Member

    Joined:
    Jun 19, 2015
    Messages:
    80
    Likes Received:
    10
    linux is most used in embedded system. not personal computer!
     
  13. dtse9

    dtse9 Member

    Joined:
    Jun 19, 2015
    Messages:
    80
    Likes Received:
    10
    is java scripting language? oh no!
     
  14. Jon Brookes

    Jon Brookes New Member

    Joined:
    Oct 10, 2015
    Messages:
    3
    Likes Received:
    4
    Java is not a scripting language. We refer to here to JavaScript - which is something entirely different

    Its gets better though, ECMAScript 6 is now a new standard for the language making it much more useful, in particular in my view for programming at the 'back end' - such as with Node.

    If you are interested in web applications, take a look at meteor, you can start to write in this new version of JavaScript pretty much straight away as it can 'understand' ECMA 6 on the serverside ( node ) anyway and it will then transpile to the client in JavaScript that the browser understands

    If your wondering where Perl fits in with all this, Perl 6 is now released and gaining momentum. There is a grant that has I believe been given recently to develop a JavaScript compiler for Perl 6 that will run on node. This could mean that some time soon you could write Perl 6 code that is compiled to JavaScript that runs on the server under nodejs and code intended to be run in browsers would be transpiled to JavaScript that would run in the browser. Wild !
     
    Last edited: Feb 22, 2016
    dtse9 likes this.
  15. dtse9

    dtse9 Member

    Joined:
    Jun 19, 2015
    Messages:
    80
    Likes Received:
    10
    sorry, i was not understood what you said. thanks a lot.
     

Share This Page