| 1 | [![Build Status](https://travis-ci.org/xandkar/beam_stats.svg?branch=master)](https://travis-ci.org/xandkar/beam_stats) |
| 2 | |
| 3 | beam_stats |
| 4 | ========== |
| 5 | |
| 6 | Periodically collects and pushes VM metrics to arbitrary consumer processes, |
| 7 | which, in-turn, can do whatever they want with the given data (such as |
| 8 | serialize and forward to some time series storage). Includes, off by default, |
| 9 | example implementations of consumers for: |
| 10 | |
| 11 | - StatsD (`beam_stats_consumer_statsd`) |
| 12 | - Graphite (`beam_stats_consumer_graphite`) |
| 13 | - CSV file (`beam_stats_consumer_csv`) |
| 14 | |
| 15 | Essentially like `folsomite`, but different. Different in the following ways: |
| 16 | |
| 17 | - More-general: consumers other than graphite can be defined |
| 18 | - More-focused: only concerned with VM metrics, while `folsomite` ships off |
| 19 | _everything_ from `folsom` (in addition to VM metrics) |
| 20 | - Easier-(for me!)-to-reason-about implementation: |
| 21 | + Well-defined metrics-to-binary conversions, as opposed to the |
| 22 | nearly-arbitrary term-to-string conversions used in `folsomite` |
| 23 | + Spec'd, tested and Dialyzed |
| 24 | - More detailed stats: |
| 25 | - **per-process**. As much process ancestry is collected as possible, then |
| 26 | anonymous processes are aggregated to their youngest-known, named |
| 27 | predecessor - this aggregation keeps the useful breadcrumbs, while |
| 28 | reducing the number of unique names from exploding, which |
| 29 | **avoids the associated problems**: |
| 30 | 1. not very useful when there're lots of short-lived processes |
| 31 | 2. exploading disk space usage in Whisper |
| 32 | - per-ETS-table |
| 33 | - and more ... see records defined in `include` directory |
| 34 | |
| 35 | For an example of using pre-process stats to track-down memory leaks, here's a |
| 36 | screenshot of the SSL connection process memory usage growth drop after upgrade |
| 37 | from 17.5 to 18.1 (back in 2015): |
| 38 | ![SSL memory leak going away](screenshot--2015-10-05--18.41.30.jpg) |
| 39 | |
| 40 | ### Adding consumers |
| 41 | |
| 42 | #### At app config time |
| 43 | |
| 44 | ```erlang |
| 45 | {env, |
| 46 | [ {production_interval , 30000} |
| 47 | , {consumers, |
| 48 | [ {beam_stats_consumer_statsd, |
| 49 | [ {consumption_interval , 60000} |
| 50 | , {dst_host , "localhost"} |
| 51 | , {dst_port , 8125} |
| 52 | , {src_port , 8124} |
| 53 | , {num_msgs_per_packet , 10} |
| 54 | |
| 55 | % If you want to name your node something other than what |
| 56 | % erlang:node() returns: |
| 57 | , {static_node_name , <<"unicorn_at_rainbow">>} |
| 58 | ]} |
| 59 | , {beam_stats_consumer_graphite, |
| 60 | [ {consumption_interval , 60000} |
| 61 | , {host , "localhost"} |
| 62 | , {port , 2003} |
| 63 | , {timeout , 5000} |
| 64 | ]} |
| 65 | , {beam_stats_consumer_csv, |
| 66 | [ {consumption_interval , 60000} |
| 67 | , {path , "beam_stats.csv"} |
| 68 | ]} |
| 69 | , {some_custom_consumer_module, |
| 70 | [ {some_custom_option_a, "abc"} |
| 71 | , {some_custom_option_b, 123} |
| 72 | ]} |
| 73 | |
| 74 | ]} |
| 75 | ]} |
| 76 | ``` |
| 77 | |
| 78 | #### Dynamically |
| 79 | |
| 80 | ```erlang |
| 81 | beam_stats_consumer:add(consumer_module, ConsumerOptions). |
| 82 | ``` |
| 83 | |
| 84 | ### Removing consumers |
| 85 | |
| 86 | Not yet implemented. |