Logstash undocumented

So I was playing around with logstash for a few days and it frustrated me that their documentation was awful. These guys really need to get someone to write more of the usability of the utility. For those who are not aware of this awesome tool, logstash, please do check out it's homepage. It is used for analysis of log files (in servers in most cases). So what is does is essentially 3 steps:

1. Collects logs by monitoring files or sockets, executing commands, etc. It has a host of input plugins, check out the docs page for a very sparse view.
2. Apply filters on the inputs, modify the way the input looks like eventually.
3. Output filters, dump the data to a socket, webapp, queues, etc. 

Logstash is generally used in cases of elastic search (e.g. kibana) that shows up neat graphs and searching can done in this data as well.

So in my particular case what I was trying to do was send data across to graphite, which just shows the data in a neat graph.

Okay, so problem no. 1. I was using a grok filter to get some data, with this filter:

{\"timeSpent\":%{NUMBER: abc},.*

Which, should ideally store the parsed number into the variable "abc" right? 

Wrong! Strange how the space will affect the decision making capability of grok and what we need is this:

{\"timeSpent\":%{NUMBER:abc},.*

Now  it works! There is a neat tool to simulate what grok wants at the sources section below.

So next, what I needed was the hostname of the box where logstash was running. Again zero in the docs. However we have a variable named: 

@source_host 

This is part of the metadata for logstash. I tried finding a list of this, but well, looks like there is none. Can someone prove me wrong, please?

Now I needed to filter some stuff out of the entire FQDN. This time the docs pointed me to a filter called mutate (gsub).

So this is how it was arranged in the filter section of the conf file now:

filter {
        mutate {
                type => "stdin-type"
                gsub => [ "@source_host", "", "" ]

        }
}

 Viola! Hoping this gives some sort of a reference for those lost at sea.

Source(s): 

1. Logstash google group post where I found the gsub way
2. Tool to check syntax of grok (select named captures only to get the variables you have used)