README.md

   1 khatus
   2 ======
   3 ![mascot](mascot.jpg)
   4
   5 Experimental system-monitor and status (bar) reporter I use with
   6 [dwm](https://dwm.suckless.org/) on GNU/Linux.
   7
   8 ![screenshot](screenshot.jpg)
   9
  10 Usage
  11 -----
  12
  13 In my `~/.xinitrc` I have something like the following:
  14
  15 ```sh
  16 ( $BIN/khatus \
  17     --wifi_interface 'wlp3s0' \
  18 | stdbuf -o L tee \
  19     >(stdbuf -o L "$BIN"/khatus_bar \
  20         -v Opt_Mpd_Song_Max_Chars=10 \
  21         -v Opt_Net_Interfaces_To_Show=wlp3s0 \
  22         -v Opt_Pulseaudio_Sink=0 \
  23     | "$BIN"/khatus_actuate_status_bar_to_xsetroot_name \
  24     ) \
  25     >(stdbuf -o L "$BIN"/khatus_monitor_energy \
  26     | "$BIN"/khatus_actuate_alert_to_notify_send \
  27     ) \
  28     >(stdbuf -o L "$BIN"/khatus_monitor_errors \
  29     | "$BIN"/khatus_actuate_alert_to_notify_send \
  30     ) \
  31 ) \
  32 2> >($BIN/twrap.sh >> $HOME/var/log/khatus/main.log) \
  33 1> /dev/null \
  34 &
  35 ```
  36 (where `twrap` is a simple script which prefixes a timestamp to each line)
  37
  38 The idea is to support appending any number of ad-hoc, experimental monitors by
  39 giving maximum flexibility for what to do with the sensor outputs, while
  40 maintaining some uniformity of msg formats (again, to ease ad-hoc combinations
  41 (e.g. Does the CPU get hotter when MPD is playing Wu-Tang?)).  `khatus_bar`,
  42 `khatus_monitor_energy` and `khatus_monitor_errors` are just some initial
  43 examples.
  44
  45 Design
  46 ------
  47
  48 ### 2.0
  49
  50 In an effort to simplify the components and their interfaces, I removed the
  51 concept of a global controller from the previous design (which, at least for
  52 now, is superfluous), so now it is essentially a pub-sub - parallel publishers
  53 (sensors) write to a pipe, which is then copied to any number of interested
  54 subscribers that can filter-out what they need and then do whatever they want
  55 with the data. Status bar is one such subscriber:
  56
  57 `P1 > pipe&; P2 > pipe&; ... PN > pipe&; tail -f pipe | tee >(S1) >(S2) ... >(SN) > /dev/null`
  58
  59 The cool thing is that, because the pipe is always read (`tail -f ... > /dev/null`),
  60 the publishers are never blocked, so we get a live stream of events to which we
  61 can attach any number of interested subscribers (` ... tee ... `) and, because
  62 the pipe is named, if a subscriber needs to - it too can publish something to
  63 the pipe without being blocked.
  64
  65 ```
  66 parallel    +----------+  +----------+          +----------+
  67 stateless   | sensor_1 |  | sensor_2 |    ...   | sensor_n |
  68 collectors  +----------+  +----------+          +----------+
  69                  |             |           |         |
  70                data          data        data      data
  71                  |             |           |         |
  72                  V             V           V         V
  73 multiplexing     +-------------+-----------+---------+
  74 to a pipe                      |
  75                                |
  76                                V
  77 copying to       +-------------+-+---------+---------+
  78 subscribers      |               |         |         |
  79                  V               V         V         V
  80               +------------+         ...      +----------------+
  81 any number of | status bar |                  | energy monitor |
  82 parallel      +------------+                  +----------------+
  83 subscribers      |                                    |
  84                  V                                    V
  85               +----------------+              +-------------+
  86               | xsetroot -name |              | notify-send |
  87               +----------------+              +-------------+
  88 ```
  89
  90 ### 1.0
  91
  92 This was an improvement of having everything in one script, but the controller
  93 was still way too complicated for no good reason.
  94
  95 ```
  96 parallel    +----------+  +----------+          +----------+
  97 stateless   | sensor_1 |  | sensor_2 |    ...   | sensor_n |
  98 collectors  +----------+  +----------+          +----------+
  99                  |             |           |         |
 100                data          data        data      data
 101                  |             |           |         |
 102                  V             V           V         V
 103 serial      +----------------------------------------------+
 104 stateful    |                controller                    |
 105 observer    +----------------------------------------------+
 106                                |
 107                        decision messages
 108 decision                       |
 109 messages                       |
 110 copied to                      |
 111 any number                     |
 112 of interested                  |
 113 filter/actuator                |
 114 combinations                   |
 115                                |
 116                                V
 117                  +-------------+-+---------+---------+
 118                  |               |         |         |
 119                  V               V         V         V
 120 parallel    +------------+ +------------+     +------------+
 121 stateless   | filter_1   | | filter_2   | ... | filter_n   |
 122 filters     +------------+ +------------+     +------------+
 123                  |               |         |         |
 124                  V               V         V         V
 125 parallel    +------------+ +------------+     +------------+
 126 stateless   | actuator_1 | | actuator_2 | ... | actuator_n |
 127 executors   +------------+ +------------+     +------------+
 128                  |              |          |         |
 129               commands       commands   commands  commands
 130                  |              |          |         |
 131                  V              V          V         V
 132             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 133             ~~~~~~~~~~~~~ operating system ~~~~~~~~~~~~~~~~~
 134             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 135 ```
 136
 137 ### 0.x
 138
 139 A single script, re-executed in a loop at some intervals, serially grabbing all
 140 the needed data and outputting a status bar string, then passed to `xsetroot -name`,
 141 while saving state in files (e.g. previous totals, to be converted to deltas).
 142
 143 This actually worked surprisingly-OK, but had limitations:
 144
 145 - I use an SSD and want to minimize disk writes
 146 - not flexible-enough to support my main goal - easy experimentation with
 147   various ad-hoc monitors:
 148     - I want to set different update intervals for different data sources
 149     - I don't want long-running data collectors to block the main loop
 150
 151 ### Actuator
 152 Actuator is anything that takes action upon controller messages. A few generic
 153 ones are included:
 154
 155 - `khatus_actuate_alert_to_notify_send`
 156 - `khatus_actuate_status_bar_to_xsetroot_name`
 157
 158 and, by default, are left disconnected from the data feed, so if desired - need
 159 to be manually attached when starting `khatus`. See usage section.
 160
 161 ### Errors
 162 Any errors encountered by any sensor are propagated as alerts by the
 163 controller, which are in turn actualized as desktop notifications by the
 164 `khatus_actuate_alert_to_notify_send` actuator:
 165
 166 ![screenshot-self-error-propagation](screenshot-self-error-propagation.jpg)
 167
 168 TODO
 169 ----
 170
 171 - tests (design is starting to take shape, so it is time)
 172 - show how many Debian package updates are available
 173 - show how many Debian package security-updates are available
 174 - monitor disk usage rate of change and alert if suspiciously fast
 175 - bring back CPU usage monitor
 176 - actual METAR parser, to replace the flaky `metar` program
 177 - status bar templating language
 178 - retry/cache for sensors fetching flaky remote resources (such as weather)
 179 - throttling of broken sensors (constantly returns errors)
 180 - alert specification language
 181     - trigger threshold
 182     - above/bellow/equal to threshold value
 183     - priority
 184     - snooze time (if already alerted, when to re-alert?)
 185     - text: subject/body
 186 - monitor processes
 187     - zombies
 188     - CPU hogs
 189     - memory hogs
 190     - memory leaks (if some process consistently grows)
 191 - report detailed status upon request (to a terminal)
 192     - use color to indicate age of data
 193
 194 Redesign notes
 195 --------------
 196
 197 - controller should not do formatting
 198 - need in-memory db for diskless feedback/throttling and cache
 199 - decouple sensor execution from sleep, i.e. a sensor is blocked not by sleep
 200   process directly, but by reading of a pipe, to where a sleep process will
 201   write a message announcing interval completion and thus signaling execution.
 202   This will allow us to manually signal a sensor to update (concretely - I just
 203   openned my laptop from sleep and want to force the weather to update
 204   immediately); likewise, the sleep process should be blocked on pipe-read
 205   until sensor execution is complete - this will allow us to reconfigure
 206   intervals at runtime (which seems like a better idea than the above in-memory
 207   DB one).
 208
 209 Ideas
 210 -----
 211
 212 - store data with rrdtool
 213 - monitor tracking numbers (17track should be easiest to get started with)
 214 - monitor stock prices
 215 - monitor some item price(s) at some store(s) (Amazon, etc.)
 216 - monitor eBay auctions (https://en.wikipedia.org/wiki/EBay_API)
 217 - monitor PayPal (https://www.programmableweb.com/api/paypal)
 218 - monitor bank account balance and transactions
 219     - https://communities.usaa.com/t5/Banking/Banking-via-API-Root/m-p/180789/highlight/true#M50758
 220     - https://plaid.com/
 221     - https://plaid.com/docs/api/
 222     - https://plaid.com/docs/api/#institution-overview
 223     - https://github.com/plaid
 224     - https://www.bignerdranch.com/blog/online-banking-apis/
 225 - monitor/log road/traffic conditions
 226     - travel times for some route over a course of time
 227     - https://msdn.microsoft.com/en-us/library/hh441725
 228     - https://cloud.google.com/maps-platform/
 229     - https://cloud.google.com/maps-platform/routes/
 230     - https://developer.mapquest.com/documentation/traffic-api/
 231     - https://developer.here.com/api-explorer/rest/traffic/traffic-flow-bounding-box
 232 - monitor news sources for patterns/substrings
 233     - http://developer.nytimes.com/
 234     - https://news.ycombinator.com/
 235     - https://lobste.rs/
 236     - https://www.undeadly.org/
 237     - http://openbsdnow.org/
 238     - https://lwn.net/
 239 - monitor a git repository
 240     - General
 241         - total branches
 242         - age of last change per branch
 243         - change set sizes
 244     - GitHub
 245         - pull requests
 246         - issues
 247 - monitor CI
 248     - Travis
 249     - Jenkins
 250 - pull/push data from/to other monitoring systems (Nagios, Graphite, etc.)
 251 - monitor file/directory age (can be used for email and other messaging systems)
 252 - monitor mailboxes for particular patterns/substrings
 253 - monitor IRC server(s)/channel(s) for particular patterns/substrings (use `ii`)
 254 - monitor iptables log
 255     - auto-(un)block upon some threshold of violations
 256 - monitor changes in an arbitrary web resource
 257     - deletions
 258     - insertions
 259     - delta = insertions - deletions
 260 - monitor/log LAN/WAN configurations (address, router, subnet)
 261 - monitor/log geolocation based on WAN IP address
 262 - correlate iptables violations with network/geolocation
 263 - monitor vulnerability databases
 264     - https://nvd.nist.gov/
 265     - https://vuldb.com/
 266     - http://cve.mitre.org/
 267 - browse https://www.programmableweb.com/ for some more ideas
 268 - GC trick: instead of actually doing GC, do a dummy run of building a status
 269   bar at `BEGIN`, to fill-in the atimes for keys we need, then use the atimes
 270   keys to build a regular expression to accept messages only from keys we
 271   actually use
 272
 273 Many of the above will undoubtedly need non-standard-system dependencies
 274 (languages, libraries, etc.), in which case - would they be better off as
 275 separate projects/repos?
 276
 277 With all these ideas, it is starting to sound very noisy, but no worries - to
 278 quickly and temporarily shut everything up - just kill `dunst` and or toggle
 279 the status bar (`Alt` + `B` in `dwm`). For a permanent change - just don't
 280 turn-on the unwanted monitors/sensors.