README.md

   1 [![Build Status](https://travis-ci.org/xandkar/beam_stats.svg?branch=master)](https://travis-ci.org/xandkar/beam_stats)
   2
   3 beam_stats
   4 ==========
   5
   6 Periodically collects and pushes VM metrics to arbitrary consumer processes,
   7 which, in-turn, can do whatever they want with the given data (such as
   8 serialize and forward to some time series storage). Includes, off by default,
   9 example implementations of consumers for:
  10
  11 - StatsD (`beam_stats_consumer_statsd`)
  12 - Graphite (`beam_stats_consumer_graphite`)
  13 - CSV file (`beam_stats_consumer_csv`)
  14
  15 Essentially like `folsomite`, but different. Different in the following ways:
  16
  17 - More-general: consumers other than graphite can be defined
  18 - More-focused: only concerned with VM metrics, while `folsomite` ships off
  19   _everything_ from `folsom` (in addition to VM metrics)
  20 - Easier-(for me!)-to-reason-about implementation:
  21     + Well-defined metrics-to-binary conversions, as opposed to the
  22       nearly-arbitrary term-to-string conversions used in `folsomite`
  23     + Spec'd, tested and Dialyzed
  24 - More detailed stats:
  25     - **per-process**. As much process ancestry is collected as possible, then
  26       anonymous processes are aggregated to their youngest-known, named
  27       predecessor - this aggregation keeps the useful breadcrumbs, while
  28       reducing the number of unique names from exploding, which
  29       **avoids the associated problems**:
  30         1. not very useful when there're lots of short-lived processes
  31         2. exploading disk space usage in Whisper
  32     - per-ETS-table
  33     - and more ... see records defined in `include` directory
  34
  35 For an example of using pre-process stats to track-down memory leaks, here's a
  36 screenshot of the SSL connection process memory usage growth drop after upgrade
  37 from 17.5 to 18.1 (back in 2015):
  38 ![SSL memory leak going away](screenshot--2015-10-05--18.41.30.jpg)
  39
  40 ### Adding consumers
  41
  42 #### At app config time
  43
  44 ```erlang
  45 {env,
  46   [ {production_interval , 30000}
  47   , {consumers,
  48       [ {beam_stats_consumer_statsd,
  49           [ {consumption_interval , 60000}
  50           , {dst_host             , "localhost"}
  51           , {dst_port             , 8125}
  52           , {src_port             , 8124}
  53           , {num_msgs_per_packet  , 10}
  54
  55           % If you want to name your node something other than what
  56           % erlang:node() returns:
  57           , {static_node_name     , <<"unicorn_at_rainbow">>}
  58           ]}
  59       , {beam_stats_consumer_graphite,
  60           [ {consumption_interval , 60000}
  61           , {host                 , "localhost"}
  62           , {port                 , 2003}
  63           , {timeout              , 5000}
  64           ]}
  65       , {beam_stats_consumer_csv,
  66           [ {consumption_interval , 60000}
  67           , {path                 , "beam_stats.csv"}
  68           ]}
  69       , {some_custom_consumer_module,
  70           [ {some_custom_option_a, "abc"}
  71           , {some_custom_option_b, 123}
  72           ]}
  73
  74       ]}
  75   ]}
  76 ```
  77
  78 #### Dynamically
  79
  80 ```erlang
  81 beam_stats_consumer:add(consumer_module, ConsumerOptions).
  82 ```
  83
  84 ### Removing consumers
  85
  86 Not yet implemented.