Hartwork Blog

We had a case at work were multi-line output of a command should be turned into an array of lines. Here’s one way to do it.

Two Bash features take part with this approach:

• $'....' syntax (a dollar right in front of a single-tick literal) activates interpolation of C-like escape sequences (see below)
• Bash variable IFS — the internal field separator affecting the way Bash applies word splitting — is temporarily changed from default spaces-tabs-and-newlines to just newlines so that we get one array entry per line

Let me demo that:

# f() { echo $'one\ntwo  spaces' ; }

# f
one
two  spaces

# IFS=$'\n' lines=( $(f) )

# echo ${#lines[@]}
2

# echo "${lines[0]}"
one

# echo "${lines[1]}"
two  spaces

I wanted to write this post for a while now. It’s about how different IT companies are, in Berlin. I find that interesting. The examples include what I have witnessed myself, at jobs but also interviews, and also tales from friends. I will leave the implications that this diversity has to you. I expect to revise this post a few times in the future. Also, splitting things into disjoint categories turned out to be difficult, so it’s somewhat arbitrary. Order does not necessarily reflect importance.

With that being said, these are some examples for ways in which Berlin IT companies differ:

People

• Diverse mix —vs— 95% locals
• Daily English —vs— German all day long
• Cheap junior labor —vs— mostly seniors
• Non-IT people and IT mostly lunch together —vs— hardly ever

Development

• Git —vs— Mercurial
• CI on master branch —vs— on all branches —vs— zero CI
• master branch always green —vs— in constant need of repair
• master branch has a single gatekeeper —vs— it has not
• Code review is waving through —vs— code review is collaboration, exchange, learning —vs— code review is not done
• Paid IDE —vs— using free edition of commercial product

Responsibilities

• Dedicated Ops people —vs— devs doing ops work as well
• Getting informed —vs— constant need to hunt information yourself
• Lots of freedom —vs— work drone with roadmap shoved down your throat without ever asking input
• Growing software —vs— getting tickets done any way

Working hours

• 32h per week welcome/tolerated —vs— 40h per week enforced
• Official core hours from 10:00 to 15:00 (5h) —vs— 10:00 to 17:00 (7h!) —vs— 9:00 to 17:00 (8h!) —vs— …
• Core hours enforced/complained —vs— come in at 11:00, we’re good
• Sleep at night —vs— be called by Ops for help at 1am
• Home office by choice —vs— home office by explicit approval

Compensation

• Salary negotiated to the ground —vs— accepting what you asked
• Salary raised every 12 months by contract —vs— no plans to raise before you bring it up
• Vacation days negotiable —vs— 25/26/28 vacation days across the company
• Overtime compensated —vs— overtime not compensated

Contract itself

• Can be adjusted —vs— refusing to change anything “for administration reasons” (to then find out …)
• Is explained when you have questions about details —vs— is not explained
• Is German and English side by side —vs— German only
• Is mostly okay —vs— raises more eyebrows than you have

Rooms

• Sales people have calls right next to you —vs— on the hallway —vs— nowhere near
• 2 people per room —vs— 4 to 6 people per room —vs— 20+ people per room
• Headphones, coming early, staying later to get work done —vs— healthy amount of distractions

Office

• No plants —vs— externally managed alibi plants —vs— 800 Euro budget to go buy some as a team next week
• Rather dark —vs— lots of light
• Windows to open —vs— no window that would open in the building

Surroundings

• Variety of tasty lunch options nearby —vs— hardly any

Equipment

• Freely chosen —vs— presets —vs— same reflecting Apple box for everyone
• Healthy chair granted —vs— on demand —vs— they’d rather have you quit
• Motor-lift table —vs— screw-left table —vs— fixed height on (of unfit/unhealthy height)

Drinks / Food

• Machine-made sparkling tap water —vs— three types bottled mineral that are hardly ever run out
• Water, wide variety of soft drinks, juices and beer —vs— two boring options
• pay-yourself sweets —vs— free sweets —vs— no sweets
• Barely drinkable coffee from press to a single button —vs— “Siebträger” espresso machine for handmade gourmet coffee (i.e. the “Gentoo way”)

Your mileage varies? Interesting differences missing? Surprised? Drop me a mail!

Version 2018-05-04

I recently dockerized a small Django application. I build the Dockerfile in a way that the resulting image would allow running the container as if it was plain manage.py, e.g. that besides docker-compose up I could also do:

# For a psql session into the database:
docker-compose run <image_name> dbshell

# Or, to run the test suite:
docker-compose run <image_name> test

To make that work, I made this Docker entrypoint script:

#! /bin/bash
# Copyright (C) 2018 Sebastian Pipping <sebastian@pipping.org>
# Licensed under CC0 1.0 Public Domain Dedication.
# https://creativecommons.org/publicdomain/zero/1.0/

set -e
set -u

RUN() {
    ( PS4='# ' && set -x && "$@" )
}

RUN wait-for-it "${POSTGRES_HOST}:${POSTGRES_PORT}" -t 30

cd /app

if [[ $# -gt 0 ]]; then
    RUN ./manage.py "$@"
else
    RUN ./manage.py makemigrations
    RUN ./manage.py migrate
    RUN ./manage.py createcustomsuperuser  # self-made

    RUN ./manage.py runserver 0.0.0.0:${APP_PORT}
fi

Management command createcustomsuperuser is something simple that I built myself for this very purpose: Create a super user, support scripting, accept a passwords as bad as “password” or “demo” without complaints, and be okay if the user exists with the same credentials already (idempotency). I uploaded createcustomsuperuser.py as a Gist to GitHub as it’s a few lines more.

Back to the entrypoint script. For the RUN ./manage.py "$@" part to work, in the Dockerfile both ENTRYPOINT and CMD need to use the [..] syntax, e.g.:

ENTRYPOINT ["/app/docker-entrypoint.sh"]
CMD []

For more details on ENTRYPOINT quirks like that I recommend John Zaccone’s well-written article “ENTRYPOINT vs CMD: Back to Basics“.

Say you have created a favicon.ico for your website and want to serve that file outside of the usual /static/images prefix, at /favicon.ico. Other favicon approaches use a code-30x redirect on HTTP level. I would rather save that extra request. Here is what I ended up with:

import os

from django.conf import settings
from django.views.static import serve

urlpatterns += [
    url(r'^(?P<path>favicon\.ico)$', serve, name='favicon',
        kwargs={'document_root': os.path.join(settings.STATIC_ROOT,
                                              'images')}),
]

Licensed under CC0 1.0 Public Domain Dedication.

Probably not new but was new to me: Just ran into this Larry the Cow tattoo online:

Larry the Gender Challenged Cow

When system-wide pip turns out too old (e.g. for lacking support for pip check), one may end up trying to update pip using a command like:

sudo pip install --upgrade pip

That’s likely to end up with this message:

Not uninstalling pip at /usr/lib/python2.7/dist-packages, owned by OS

That non-error and the confusion that easily happens right after is why I’m writing this post.

So let’s look at the whole thing in a bit more context on a shell, a Debian jessie one in this case:

# cat /etc/debian_version 
8.10

# pip install --upgrade pip ; echo $?
Downloading/unpacking pip from https://pypi.python.org/packages/b6[..]44
  Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB): 1.3MB downloaded
Installing collected packages: pip
  Found existing installation: pip 1.5.6
    Not uninstalling pip at /usr/lib/python2.7/dist-packages, owned by OS
Successfully installed pip
Cleaning up...
0

# pip --version
pip 1.5.6 from /usr/lib/python2.7/dist-packages (python 2.7)

Now the interesting part is that it looks like pip would not have been updated. That impression is false: Latest pip has been installed successfully (to /usr/local/bin). One of two things is going on here:

a) Unexpected Path resolution order

You have /usr/bin/ before /usr/local/bin/ in $PATH, e.g. as with root of Debian jessie, so that the new pip has no chance of winning the race of path resolution for pip. For example:

# sed 's,:,\n,g' <<<"$PATH"
/bin
/sbin
/usr/bin
/usr/sbin
/usr/local/bin
/usr/local/sbin
/opt/bin
/usr/lib/llvm/5/bin
/usr/lib/llvm/4/bin

b) Location hashing at shell level

Your shell has hashed the old location of pip (as Bash would do) and “hides” the new version from you in the current shell session. To see that in action, we utilize Bash builtins type and hash:

# type pip
pip is hashed (/usr/bin/pip)

# pip --version
pip 1.5.6 from /usr/lib/python2.7/dist-packages (python 2.7)

# hash -d pip

# type pip
pip is /usr/local/bin/pip

# pip --version
pip 9.0.1 from /usr/local/lib/python2.7/dist-packages (python 2.7)

So in either case you can run a recent pip from /usr/local/bin/pip right after pip install --upgrade pip, no need to resort to get-pip.py or so, in fact.

This “crazy” vulnerability in LibreOffice only came to my attention recently:

LibreOffice < 6.0.1 – ‘=WEBSERVICE’ Remote Arbitrary File Disclosure (exploit-db.com)

Please make sure yours peers update in time.

Some people care if software is free of cost or if it has the best features, above everything else. I don’t. I care that I can legally inspect its inner workings, modify and share modified versions. That’s why I happily avoid macOS, Windows, Skype, Photoshop.

I ran into these two pieces involving Gentoo in the Gallery of Free Software lovers and would like to share them with you:

Images are licensed under CC BY-SA 4.0 (with attribution going to Free Software Foundation Europe) as confirmed by Max Mehl.

Hi!

After moving uriparser from SourceForge to GitHub and re-creating its website from scratch earlier this year, today uriparser 0.8.5 has been released and is available for download.

Quoting from the change log, the changes are as follows:

2018-02-07 -- 0.8.5

  * Changed: The uriparser project has moved from SourceForge to GitHub:
      Code + issue tracker: https://github.com/uriparser/uriparser
      New website: https://uriparser.github.io/
      Please update any links of yours, accordingly. Thank you!
  * Fixed: Memleak in out-of-memory clean-up code
      of URI normalization, related to SF.net bug #28.
      Thanks to Chris Hills for the report!
  * Fixed: Fix compilation of uriparse(1) on FreeBSD
      Thanks to Ed Schouten for the patch!
  * Fixed: Fix C90 compilation errors
      Thanks to Joel Cunningham for the patches!
  * Fixed: Space requirements documented for uriWindowsFilenameToUriStringA
      given URI "file://server1/file1.txt" (SF.net bug #31)
      Thanks to threedyd for the report!
  * Fixed: Compiler warnings
      Thanks to Joel Cunningham for the patches!
  * Fixed: Stop exporting internal function RemoveBaseUriImpl
      Thanks to Joel Cunningham for the report!
  * Fixed: API documentation front page no longer empty with Doxygen 1.8.13
  * Fixed: "make -C doc install" fixed for lack of .map files
  * Improved: Communicate that absolutePath is always URI_FALSE for URIs
      with a host in uriparse CLI tool output and Uri.h header
      (GitHub #2, SF.net #30)
  * Soname: 1:21:0

It would be cool if you could update any uriparser you have power to update and report any issues you run into. Thanks a bunch!

Best

Sebastian

I “know someone” who took way to long to find an answer to the question how to disable binary logging with MySQL so that it would no longer fills up all disk space putting a certain instance of WordPress offline every now and then. I find this solution rather unexpected: Putting

[mysqld]
skip-log-bin

into /etc/mysql/my.cnf is a weird mix of ini-file syntax [mysqld] with non-ini file syntax skip-log-bin — without any “ = value” assignment. To be sure that binary logging is turned off after a restart of the service, here’s a way to check:

mysql> SHOW VARIABLES LIKE 'log_bin';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| log_bin       | OFF   |
+---------------+-------+
1 row in set (0.00 sec)

Finally!

If this post turns wrong in the future, it did work for MySQL 5.6.39.