Eikonal Blog

2011.09.30

awk

Filed under: scripting, unix — Tags: , — sandokan65 @ 11:03

Passing shell variables to AWK

Thing that works well for me: 

    awk '{print "'"$VARIABLE"'"}' 1 > 2

Related here: Scripting languages – https://eikonal.wordpress.com/2010/06/15/awk-sed/ | Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/ | SED tricks – https://eikonal.wordpress.com/2010/10/05/sed-tricks/ | Memory of things disappearing > nmap stuff > getports.awk – https://eikonal.wordpress.com/2010/06/23/memory-of-things-disappearing-nmap-stuff-getports-awk/

2010.07.13

Command line based text replace

Filed under: perl, transformers, unix — Tags: , , , , , , , , , , — sandokan65 @ 13:36

sed

  • sed 's/Mark Monre/Marc Monroe/' 1.txt > 2.txt
  • find ./* -type f -exec sed -i 's///g' {} \;

The “replace” command

  • Syntax: 
    replace OLD-STRING NEW-STRING OUTPUT-FILE
  • Example: 
    $ replace UNIX Linux  newfile
  • Example: 
    $ cat /etc/passwd | replace : '|'
  • Partial support for regular expressions: \^ – matches start of line, and $ matches end of line.
  • Example: replace all IP address 192.168.1.2 start of line: 
    $ replace \^192.168.1.2 192.168.5.10  newfile
  • a bash script, ‘fixer.sh’
    #!/bin/bash
replace CHANGEFROM CHANGETO $1.tmp
rm $1
mv $1.tmp $1

    now run this command line: 

    $ grep CHANGEFROM |cut -d':' -f1 |xargs -n 1 fixer.sh

    the results is that all files in the directory (or whatever you grep for) will be changed automagically.
    just make sure the grep doesn’t include the fixer script itself, or it will die half-way through changing when execute permissions are reset!

Perl

Sources:

Related: Regular expressions – https://eikonal.wordpress.com/2010/04/02/regular-expressions/ | Perl online – https://eikonal.wordpress.com/2010/02/15/perl-online/

2010.06.15

Scripting languages

Filed under: scripting, unix — Tags: , , , , , , — sandokan65 @ 14:35

Unix shells:

Windows Scripting:

Multi-language pages:

Compiling shell scripts

Can not be done.

Related here: Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/ | SED tricks – https://eikonal.wordpress.com/2010/10/05/sed-tricks/ | Memory of things disappearing > nmap stuff > getports.awk – https://eikonal.wordpress.com/2010/06/23/memory-of-things-disappearing-nmap-stuff-getports-awk/ | AWK – https://eikonal.wordpress.com/2011/09/30/awk/

2010.04.02

Regular expressions

Filed under: regexp, scripting, unix — Tags: , , , , , , , , , , , , , , , , — sandokan65 @ 14:52

Sites

Tools

Standalone tools:

Online testers:

Books

Tidbits

Sources: The above links.

  • [abc] – A single character: a, b or c
  • [^abc] – Any single character but a, b, or c
  • [a-z] – Any single character in the range a-z
  • [a-zA-Z] – Any single character in the range a-z or A-Z
  • ^ – Start of line
  • $ – End of line
  • \A – Start of string
  • \z – End of string
  • . – Any single character
  • \s – Any whitespace character
  • \S – Any non-whitespace character
  • \d – Any digit
  • \D – Any non-digit
  • \w – Any word character (letter, number, underscore)
  • \W – Any non-word character
  • \b – Any word boundary character
  • (…) – Capture everything enclosed
  • (a|b) – a or b
  • a? – Zero or one of a
  • a* – Zero or more of a
  • a+ – One or more of a
  • a{3} – Exactly 3 of a
  • a{3,} – 3 or more of a
  • a{3,6} – Between 3 and 6 of a
  • ^\s[ \t]*$ – Match a blank line
  • \d{2}-\d{5} – Validate an ID number consisting of 2 digits, a hyphen, and another 5 digits

Special common strings:

  • Personal Name: ^[\w\.\’]{2,}([\s][\w\.\’]{2,})+$
  • Username: ^[\w\d\_\.]{4,}$
  • Password at least 6 symbols: ^.{6,}$
  • Password or empty input: ^.{6,}$|^$
  • email: ^[\_]*([a-z0-9]+(\.|\_*)?)+@([a-z][a-z0-9\-]+(\.|\-*\.))+[a-z]{2,6}$
  • Email address: \b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b[A-z0-9_.%+-]+@[A-z0-9_.%+-]+\.[A-z]{2,4}
  • US phone: \W?\d{3}\W?\d{3}\W?\d{4}
  • US Phone number: ^\+?[\d\s]{3,}$
  • US Phone with code: ^\+?[\d\s]+\(?[\d\s]{10,}$
  • URL: \W?\d{3}\W?\d{3}\W?\d{4}\b\w+://(\w|-|\.|/)+(/|\b)
  • US Social Security Number (SSN): \d{3}-\d{2}-\d{4}
  • US ZIP: \d{5}(-\d{4})?
  • IP (v4) address: \b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b
  • IP (v4) address: \b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b
  • IP (v4) address: ^(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5]){3}$
  • IP (v4) address: \b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b
  • IP (v4) address: \b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b
  • IP (v6) address:
  • MAC address: ^([0-9a-fA-F][0-9a-fA-F]:){5}([0-9a-fA-F][0-9a-fA-F])$
  • Positive Integers: ^\d+$
  • Negative Integers: ^-\d+$
  • Integer: ^-{0,1}\d+$
  • Positive Number: ^\d*\.{0,1}\d+$
  • Negative Number: ^-\d*\.{0,1}\d+$
  • Positive Number or Negative Number: ^-{0,1}\d*\.{0,1}\d+$
  • Floating point number: [-+]?([0-9]*\.[0-9]+|[0-9]+)
  • Floating point number: [-+]?(?:\b[0-9]+(?:\.[0-9]*)?|\.[0-9]+\b)(?:[eE][-+]?[0-9]+\b)?
  • Roman number: ^(?i:(?=[MDCLXVI])((M{0,3})((C[DM])|(D?C{0,3}))?((X[LC])|(L?XX{0,2})|L)?((I[VX])|(V?(II{0,2}))|V)?))$
  • Domain Name: ^([a-zA-Z0-9]([a-zA-Z0-9\-]{0,61}[a-zA-Z0-9])?\.)+[a-zA-Z]{2,6}$
  • Domain Name: ^([a-z][a-z0-9\-]+(\.|\-*\.))+[a-z]{2,6}$
  • Windows File Name: (?i)^(?!^(PRN|AUX|CLOCK\$|NUL|CON|COM\d|LPT\d|\..*)(\..+)?$)[^\\\./:\*\?\”\|][^\\/:\*\?\”\|]{0,254}$
  • Date in format yyyy-MM-dd: (19|20)\d\d([- /.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])
  • Date (dd mm yyyy, d/m/yyyy, etc.): ^([1-9]|0[1-9]|[12][0-9]|3[01])\D([1-9]|0[1-9]|1[012])\D(19[0-9][0-9]|20[0-9][0-9])$
  • Year 1900-2099: ^(19|20)[\d]{2,2}$

Related (here at this blog):
Command line based text replace – https://eikonal.wordpress.com/2010/07/13/command-line-based-text-replace/ |
Perl online – https://eikonal.wordpress.com/2010/02/15/perl-online/

Create a free website or blog at WordPress.com.