README.md

   1 [![Build Status](https://travis-ci.org/xandkar/beam_stats.svg?branch=master)](https://travis-ci.org/xandkar/beam_stats)
   2
   3 beam_stats
   4 ==========
   5
   6 Periodically collects and pushes VM metrics to arbitrary consumer processes,
   7 which, in-turn, can do whatever they want with the given data (such as
   8 serialize and forward to some time series storage). Includes, off by default,
   9 example implementations of consumers for:
  10
  11 - StatsD (`beam_stats_consumer_statsd`)
  12 - Graphite (`beam_stats_consumer_graphite`)
  13 - CSV file (`beam_stats_consumer_csv`)
  14
  15 Essentially like `folsomite`, but different. Different in the following ways:
  16
  17 - More-general: consumers other than graphite can be defined
  18 - More-focused: only concerned with VM metrics, while `folsomite` ships off
  19   _everything_ from `folsom` (in addition to VM metrics)
  20 - Easier-(for me!)-to-reason-about implementation:
  21     + Well-defined metrics-to-binary conversions, as opposed to the
  22       nearly-arbitrary term-to-string conversions used in `folsomite`
  23     + Spec'd, tested and Dialyzed
  24 - More detailed stats:
  25     - **per-process**. As much process ancestry is collected as possible, then
  26       anonymous processes are aggregated to their youngest-known, named
  27       predecessor - this aggregation keeps the useful breadcrumbs, while
  28       reducing the number of unique names from exploding, which
  29       **avoids the associated problems**:
  30         1. not very useful when there're lots of short-lived processes
  31         2. exploading disk space usage in Whisper
  32     - per-ETS-table
  33
  34 For an example of using pre-process stats to track-down memory leaks, here's a
  35 screenshot of the SSL connection process memory usage growth drop after upgrade
  36 from 17.5 to 18.1 (back in 2015):
  37 ![SSL memory leak going away](screenshot--2015-10-05--18.41.30.jpg)
  38
  39 ### Adding consumers
  40
  41 #### At app config time
  42
  43 ```erlang
  44 {env,
  45   [ {production_interval , 30000}
  46   , {consumers,
  47       [ {beam_stats_consumer_statsd,
  48           [ {consumption_interval , 60000}
  49           , {dst_host             , "localhost"}
  50           , {dst_port             , 8125}
  51           , {src_port             , 8124}
  52           , {num_msgs_per_packet  , 10}
  53
  54           % If you want to name your node something other than what
  55           % erlang:node() returns:
  56           , {static_node_name     , <<"unicorn_at_rainbow">>}
  57           ]}
  58       , {beam_stats_consumer_graphite,
  59           [ {consumption_interval , 60000}
  60           , {host                 , "localhost"}
  61           , {port                 , 2003}
  62           , {timeout              , 5000}
  63           ]}
  64       , {beam_stats_consumer_csv,
  65           [ {consumption_interval , 60000}
  66           , {path                 , "beam_stats.csv"}
  67           ]}
  68       , {some_custom_consumer_module,
  69           [ {some_custom_option_a, "abc"}
  70           , {some_custom_option_b, 123}
  71           ]}
  72
  73       ]}
  74   ]}
  75 ```
  76
  77 #### Dynamically
  78
  79 ```erlang
  80 beam_stats_consumer:add(consumer_module, ConsumerOptions).
  81 ```
  82
  83 ### Removing consumers
  84
  85 Not yet implemented.