Writing Regex For Qradar

Writing regex for Qradar is a pretty nifty thing; task which I enjoyed the most. Qradar uses JAVA regex engine and using the 'extract property' UI window you can define really nice and complex regex as well. In this page I would like to share my tips,techniques as well some of the limitations of using regular expression in Qradar.

TIPS and SCENARIOS

  1.  Consider you want to write regex for a log where you are interested to extract both success and login failures. What are your options? Consider sample custom log to be 
                            user abc Ip address 192.168.1.4 login successful.

                           user abc Ip address 192.168.1.4 login failure.
  
You got two options over here write one regex which caters for both of these events and distinguish each of them using the apply filter option in UI, or have two separate custom properties for each event and call them as you want in the search filter? 

Which option would you go for? What would be your consideration to choose any....

I myself would prefer option B for following good reasons:-

  • Less post-processing which in my experience means If you got to search the above events in number of 20K log events in 1 day-depending upon your appliance,  deployment model as well hw specs means as soon as you the search completes you have the right results displayed on the screen. In other case; for every x number of record fetched from /ariel there is post-processing (front java) is involved from finding in the text (payload) the search word "failure or success" depending upon how you are going to write your search filter.
For regex part ; Here is the def

login\s+(failure)

You can test it here at rubular.

 

10 comments:

  1. Sorry for placing this in the discussion section....

    ReplyDelete
    Replies
    1. Hello Andrew,
      Thanks for your interest.
      here is my email
      abdullah.halimah@gmail.com
      skypeID abdullah.halimah
      Cell# +92-333-3663088

      Delete
  2. Recipes:

    MAC Address:
    ^([0-9a-zA-Z]{2}[:-]){5}([0-9a-zA-Z]{2})$
    ([0-9a-zA-Z]{2}[:-]){5}
    ([0-9a-fA-F][0-9a-fA-F]:){5}([0-9a-fA-F][0-9a-fA-F])

    IP Address:
    \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}
    (\d(.+?).\d(.+?).\d(.+?).)
    (\d+.\d+.\d+.\d+)

    DOMAIN:
    ([a-zA-Z0-9]([a-zA-Z0-9\-]{0,61}[a-zA-Z0-9])?\.)+[a-zA-Z]{2,6}
    (\w+://\w{3}.\w+.\w{3})
    (\w+://\w{3}.\w+.\w+.\w{3})

    DATE:
    (19|20)\d\d([- /.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])
    (\d{4}/\d{2}/\d{2})

    TIME:
    (\d{2}:\d{2}:\d{2})

    EMAIL:
    ^([a-z0-9_\.-]+)@([\da-z\.-]+)\.([a-z\.]{2,6})$
    (\w+.\w+@\w+.\w{1,3})


    ReplyDelete
  3. very useful indeed

    ReplyDelete
  4. I want to know how to write a regex for multiline text from payload having next line \n after each line..e.g payload section
    SQL-TEXT: Select * from employee
    Where empid='test';

    This data has newline after employee..kindly help

    ReplyDelete
  5. so u want a regex to match more then 1 plus times. It is best you use filter custom property "contains" as last time i checked regex in qradar don't do multilines matches.

    ReplyDelete
  6. hello Abdullah Halimah can i talk to you on gtalk

    ReplyDelete
  7. Can anybody suggest me the general regular expression for IP(IPv4 and IPv6), MAC address,

    ReplyDelete
  8. https://www.ibm.com/support/knowledgecenter/SS42VS_7.2.8/com.ibm.qradar.doc/t_qradar_regex_cus_prop.html

    The following examples show sample regular expressions:

    Email: (.+@[^\.].*\.[a-z]{2,}$)
    URL: (http\://[a-zA-Z0-9\-\.]+\.[a-zA-Z]{2,3}(/\S*)?$)
    Domain Name: (http[s]?://(.+?)["/?:])
    Floating Point Number: ([-+]?\d*\.?\d*$)
    Integer: ([-+]?\d*$)
    IP address: (\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)

    ReplyDelete
  9. IPv6 Examples

    4030:00BC:0000:00A4:0267:01FF:FE01:7352
    /^([0-9A-Fa-f]{4}:){7}[0-9A-Fa-f]{4}$/

    sedecimal notation: 4030:B:0:A4:267:1F:0:52
    /^([0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}$/

    A complex one...
    ::4030:A4:0:2:7352 ; :: ; 2030:34E3::8CF3:7623
    /^(([0-9A-Fa-f]{1,4}:){1,7}|:)(:|([0-9A-Fa-f]{1,4}:){1,7})$/

    ReplyDelete