2020, week 47

Created: by Pradeep Gowda Updated: Nov 22, 2020 Tagged: weekly

This is the week 47 of 2020. Nov 16-22. Last week

[TOC]

This weekly section is already seeing consistent “sections” like software, code, links etc. Some more sections to add here – reading (specifically around the idea of “cluster of 5 books and 40 pages a day”).

There are two ways to observe your personal “journey”:

  1. observe things as they actually happen, record them and observe
  2. have a system for making progress in areas that you deem are important, record what you planned, what was actually done, and the delta between the two and observe

The ISO8601 definition of the week of the year is different than the one you get from Python

>>> import time
>>> print(time.strftime('%U'))
46
# we dont want this

The way to get the same using the date program: date +%U. In emacs lisp:

(concat (file-name-as-directory btbytes-directory)
    (format "content/posts/%s-w%s.md"
      (format-time-string "%Y")
      (format-time-string "%U")))

Overcommunicating, and writing well is key to success in modern workplaces. This is especially true for remote working. Fortunately, writing well, or atleast writing better over time can be result from overcommunicating – in writing!. Couple of blog posts on writing in software careers: Being Glue and mattermost

Patrick Jane’s character in The Mentalist is always making tea. So, I started looking at “propah” ways of making “dip-dip” tea (not chaha/chai). Apparently, George Orwell wrote an article about making a Nice cup of Tea

Software

Red panda

Redpanda, a kafka compatible streaming platform without the need for ZK, using Raft consensus protocol, written in C++ on top of Seastar platform (same as ScyllaDB) was made available as “Source available” 1.

This makes it an attractive option for many projects where Kafka is too big of a dependency to manage. I have always believed that even non-Java dev teams can benefit from Kafka, but cultural and technical challenges prevent them from doing so.

Clickhouse

Clickhouse is a columnar database from Russian internet comapny - Yandex.ru. From Russia With Love: How ClickHouse Saved our Data | Mux blog talks about ” transitioning the storage of video views from a Citus cluster to a ClickHouse cluster”.

Saw this bit on their frontpage:

Total cost of ownership could be further lowered by using commodity hardware with rotating disk drives instead of enterprise grade NVMe or SSD without significant sacrifices in latency for most kinds of queries.

I wonder how much savings they are talking about if you have 100G/1TB data etc.

Linux virtualization on mac

This article on Opensource.com is the best explanation of how to run Linux on MacOS using qemu. I have been looking for this option since Virtualbox is now verboten in many circles. I’d like to try the setup with Ubuntu instead.

Cloud images for Ubuntu Focal look different (they don’t have qcow2 images for one), so I’ll have to figure out exact instructions to make that work.

MacOS Sleep Wake failure in EFI

MacOS often rebooted with this failure:

Sleep Wake failure in EFI
Failure code:: 0x0171260e 0x0000001f
Please IGNORE the below stackshot

Set sudo pmset -a standby 0 following this reddit thread:

$ sudo pmset -a standby 0
Password:
Warning: Idle sleep timings for "AC Power" may not behave as expected.
- Disk sleep should be non-zero whenever system sleep is non-zero.

Let’s see if this fixes it.

DevOps

  • “Work load orchestration” — https://www.nomadproject.io/ a competitor to Kubernetes.
  • Investigate “Podman” as an alternative to Docker.

Architecture Decision Records

This discussion around “Architecture Decision Records” (and the original linked article) is a good thread on how ADRs can benefit and also they can fail.

Stripe Engineering blog has a good primer on when to write an ADR.

Architectural Decision Records | adr.github.io is github org with templates etc.

Code

URL Shortener

Someone posted this project “This is a minimal URL shortener that can be entirely hosted on GitHub pages.”. Seems like it is using too many moving parts to accomplish a simple task.

We can do this with:

  1. a json file containing the mapping of shortcut (key) to target URL (value)
  2. an HTML file containing some javascript logic to look up k-v and redirect
  3. Optionally, we can rewrite requests to this page to make it look nice (eg: https://domain/go.html?to=gh can rewritten to https://domain.com/go/gh)

Update: Implemented the URL Shortener using just javascript + JSON + [NGINX]

Python Memory Management

Run Python Applications Efficiently With malloc_trim.

  • Limiting how much memory an application can use is controllable via using cgroups (also mentioned in autotune portion of Redpanda docs)
  • You can release free memory back to the operating system using the malloc_trim libc call.
  • This call is quite straight forward using Python’s ctypes module.

Basic algoritm described agove:

  • Check every 30-60, if memory usage by the current process is above a threshold
  • Call libc.malloc_trim() method. The method is documented here

Additional discussion over at HN.

git

“Git gets easier once you get the basic idea that branches are homeomorphic endofunctors mapping submanifolds of a Hilbert space.” – agnoster?. An explanation of that technobabble.

Project Ideas

Zed Shaw has a great thread (TR)on generating project ideas to learn, and practice coding. Many things I’m already doing or familiar with, but this one tweet stood out in particular:

  1. Finally, take anything that people love and remove something important:
  • Dating site with no images.
  • Blog with no text.
  • Twitter with no followers.
  • Game with no visuals at all, just sound. This simplifies the project, and forces you to think differently about it.

Good #ProgrammingAdvice

Mental Models

Heilmeier Catechism

George H. Heilmeier, a former DARPA director (1975-1977), crafted a set of questions known as the “Heilmeier Catechism” to help Agency officials think through and evaluate proposed research programs.

  •   What are you trying to do? Articulate your objectives using absolutely no jargon.
  •   How is it done today, and what are the limits of current practice?
  •   What is new in your approach and why do you think it will be successful?
  •   Who cares? If you are successful, what difference will it make?
  •   What are the risks?
  •   How much will it cost?
  •   How long will it take?
  •   What are the mid-term and final “exams” to check for success?

Business

https://www.tinycapital.com/

In 2014, we stopped starting companies and started buying them using a simple one-month process, then leaving the companies alone to do their thing.

This is a good business model, one that is made popular by Bufett and co. buy “good businesses” and let the people who made the business successful in the first place, run them.

Reading

  • 97 Things Every Engineering Manager Should Know

New books:

  • Giordano Bruno’s De Umbris Idearum translated into English “On the Shadows of Ideas & The Art of Memory” by Scott Gosnell.
  • Communications of the ACM 12/2020 received.

Writing

  • investigate if finding your writing voice is easier if you use the voice to dictate and capture the initial draft instead.

Words

  • “picayune” – petty; worthless. (old usage) a small coin (half a Spanish real) of little value, especially a 5-cent piece.

Blog of the week

  1. The source is available on github, under BSL (Business Software License), which is not an “Open Source” license, but in 10 years the software will be released under Apache License v2.0. This is to prevent cloud vendors from eating their lunch.↩︎