2020, week 47
Created:
This is the week 47 of 2020. Nov 16-22. Last week
[TOC]
¶ This weekly
section is already seeing consistent “sections” like software, code,
links etc. Some more sections to add here – reading (specifically around
the idea of “cluster of 5 books
and 40 pages a day”).
¶ There are two ways to observe your personal “journey”:
- observe things as they actually happen, record them and observe
- have a system for making progress in areas that you deem are important, record what you planned, what was actually done, and the delta between the two and observe
¶ The ISO8601 definition of the week of the year is different than the one you get from Python
>>> import time
>>> print(time.strftime('%U'))
46
# we dont want this
The way to get the same using the date program:
date +%U
. In emacs lisp:
(concat (file-name-as-directory btbytes-directory)
(format "content/posts/%s-w%s.md"
(format-time-string "%Y")
(format-time-string "%U")))
¶ Overcommunicating, and writing well is key to success in modern workplaces. This is especially true for remote working. Fortunately, writing well, or atleast writing better over time can be result from overcommunicating – in writing!. Couple of blog posts on writing in software careers: Being Glue and mattermost
¶ Patrick Jane’s character in The Mentalist is always making tea. So, I started looking at “propah” ways of making “dip-dip” tea (not chaha/chai). Apparently, George Orwell wrote an article about making a Nice cup of Tea
Software
Red panda
Redpanda, a kafka compatible streaming platform without the need for ZK, using Raft consensus protocol, written in C++ on top of Seastar platform (same as ScyllaDB) was made available as “Source available” 1.
This makes it an attractive option for many projects where Kafka is too big of a dependency to manage. I have always believed that even non-Java dev teams can benefit from Kafka, but cultural and technical challenges prevent them from doing so.
Clickhouse
Clickhouse is a columnar database from Russian internet comapny - Yandex.ru. From Russia With Love: How ClickHouse Saved our Data | Mux blog talks about ” transitioning the storage of video views from a Citus cluster to a ClickHouse cluster”.
Saw this bit on their frontpage:
Total cost of ownership could be further lowered by using commodity hardware with rotating disk drives instead of enterprise grade NVMe or SSD without significant sacrifices in latency for most kinds of queries.
I wonder how much savings they are talking about if you have 100G/1TB data etc.
Linux virtualization on mac
This article on
Opensource.com is the best explanation of how to run Linux on MacOS
using qemu
. I have been looking for this option since
Virtualbox is now verboten in many circles. I’d like to try the setup
with Ubuntu instead.
Cloud images
for Ubuntu Focal look different (they don’t have qcow2
images for one), so I’ll have to figure out exact instructions to make
that work.
MacOS Sleep Wake failure in EFI
MacOS often rebooted with this failure:
Sleep Wake failure in EFI
Failure code:: 0x0171260e 0x0000001f
Please IGNORE the below stackshot
Set sudo pmset -a standby 0
following this reddit thread:
$ sudo pmset -a standby 0
Password:
Warning: Idle sleep timings for "AC Power" may not behave as expected.
- Disk sleep should be non-zero whenever system sleep is non-zero.
Let’s see if this fixes it.
DevOps
- “Work load orchestration” — https://www.nomadproject.io/ a competitor to Kubernetes.
- Investigate “Podman” as an alternative to Docker.
Architecture Decision Records
This discussion around “Architecture Decision Records” (and the original linked article) is a good thread on how ADRs can benefit and also they can fail.
Stripe Engineering blog has a good primer on when to write an ADR.
Architectural Decision Records | adr.github.io is github org with templates etc.
Code
URL Shortener
Someone posted this project “This is a minimal URL shortener that can be entirely hosted on GitHub pages.”. Seems like it is using too many moving parts to accomplish a simple task.
We can do this with:
- a json file containing the mapping of shortcut (key) to target URL (value)
- an HTML file containing some javascript logic to look up k-v and redirect
- Optionally, we can rewrite requests to this page to make it look
nice (eg:
https://domain/go.html?to=gh
can rewritten tohttps://domain.com/go/gh
)
Update: Implemented the URL Shortener using just javascript + JSON + [NGINX]
Python Memory Management
Run
Python Applications Efficiently With malloc_trim
.
- Limiting how much memory an application can use is controllable via
using
cgroups
(also mentioned inautotune
portion of Redpanda docs) - You can release free memory back to the operating system using the
malloc_trim
libc call. - This call is quite straight forward using Python’s
ctypes
module.
Basic algoritm described agove:
- Check every 30-60, if memory usage by the current process is above a threshold
- Call
libc.malloc_trim()
method. The method is documented here
Additional discussion over at HN.
git
“Git gets easier once you get the basic idea that branches are homeomorphic endofunctors mapping submanifolds of a Hilbert space.” – agnoster?. An explanation of that technobabble.
Project Ideas
Zed Shaw has a great thread (TR)on generating project ideas to learn, and practice coding. Many things I’m already doing or familiar with, but this one tweet stood out in particular:
- Finally, take anything that people love and remove something important:
- Dating site with no images.
- Blog with no text.
- Twitter with no followers.
- Game with no visuals at all, just sound. This simplifies the project, and forces you to think differently about it.
Good #ProgrammingAdvice
Mental Models
Heilmeier Catechism
George H. Heilmeier, a former DARPA director (1975-1977), crafted a set of questions known as the “Heilmeier Catechism” to help Agency officials think through and evaluate proposed research programs.
What are you trying to do? Articulate your objectives using absolutely no jargon.
How is it done today, and what are the limits of current practice?
What is new in your approach and why do you think it will be successful?
Who cares? If you are successful, what difference will it make?
What are the risks?
How much will it cost?
How long will it take?
What are the mid-term and final “exams” to check for success?
Business
In 2014, we stopped starting companies and started buying them using a simple one-month process, then leaving the companies alone to do their thing.
This is a good business model, one that is made popular by Bufett and co. buy “good businesses” and let the people who made the business successful in the first place, run them.
Reading
- 97 Things Every Engineering Manager Should Know
New books:
- Giordano Bruno’s De Umbris Idearum translated into English “On the Shadows of Ideas & The Art of Memory” by Scott Gosnell.
- Communications of the ACM 12/2020 received.
Writing
- investigate if finding your writing voice is easier if you use the voice to dictate and capture the initial draft instead.
Words
- “picayune” – petty; worthless. (old usage) a small coin (half a Spanish real) of little value, especially a 5-cent piece.
Blog of the week
Dan Luu’s blog - https://danluu.com/ is an excellent example of high signal technical blogging. The best thing about his blog is that not only the topics he writes novel, they ask interesting question, teach you something new about that field (for me personally, hardware engineering, hardware testing etc). Some good posts:
From Around the Web {: #links}
- Editing the Sanskrit Writings of Vasishtha Kavyakantha Ganapati Muni | by Sampadananda Mishra | Nov, 2020 | Medium
- What Gödel Discovered with an explanation using Lisp
- Programming blog - Perfumed nightmare on Lisp, Clojure, unix etc.
- TODO/PAGES: Talks I liked by Igal Tabachnik with talks on Haskell, Scala, functional programming, Programming languages, “makes you think”, and some classics (eg: wat, growing a language by GuySteele, the mess we are in by JoeArmstrong)
- “Work,
I guess,” he says. “I never loafed.” – Life Lessons from a
97-Year-Old Lobsterman - John Olson. Subscribed to Outside magazine on
2020-11-21 - Kanara: A Brief History of its Culture, Maritime Trade and Wars (12th -19th Century) | by Bharath Bhat | Nov, 2020 – first of the three part series.