Using GNOME Keyring in Docker Container

UPDATE 28–08–2022:

This story become obsolete as in GLib starting from 2.70 the mechanism that we’re relying into will not work. See https://gitlab.gnome.org/GNOME/gnome-keyring/-/issues/77 for more details.

END OF UPDATE

This story is dedicated of describing how Docker container can be configure to use Python package keyring. It is non-trivial task. But first of all, we need to go over some basic .

GNOME Keyring is service designed to store security credentials such as passwords, keys, certificates and so on together with a small amount of relevant metadata and make them available to applications. The sensitive data is encrypted and stored in a keyring file. GNOME Keyring — is persistent storage with keeping data on a hard disk. This is password manager similar to KeePass, KWallet, Apple Keychain, LastPass and others.

Don’t be confused with Linux keyrings that is in-kernel key management and retention facility providing the ability to store secret information. Linux kernel — is a kernel’s facility for “passwords caching” — it stores them in a computer’s memory during an active user’s/system session.

D-Bus on of the Linux kernel IPC — Inter-Process Communication — mechanisms, allowing for separate processes inside of an operating system to communicate with each other. D-Bus in general: it’s a “bus” for IPC. It can be a session bus, a system bus or private bus. Session bus is one that we’re interesting in. In the past CORBA or DCOP (DCOP is based on X Window System Inter-Client Exchange protocol) was used for this purpose. You can think about it as lightweight version of RPC/REST.

D-Bus is used for communication between application process and by GNOME Keyring. When application want to get/set security credentials it uses D-Bus to reach GNOME Keyring Daemon, more specifically it uses it’s session bus.

The Python keyring library provides an easy way to access the system keyring service from Python. It can be used in any application that needs safe password storage…

…The python keyring lib contains implementations for several backends. The library will attempt to automatically choose the most suitable backend for the current environment. Users may also specify the preferred keyring in a config file or by calling the set_keyring() function…

Using Keyring

The basic usage of keyring is pretty simple: just call keyring.set_password and keyring.get_password:

>>> import keyring
>>> keyring.set_password(“system”, “username”, “password”)
>>> keyring.get_password(“system”, “username”) ‘password’

Command-line Utility

Keyring supplies a keyring command which is installed with the package. After installing keyring in most environments, the command should be available for setting, getting, and deleting passwords. For more information on usage, invoke with no arguments or with --help as so:

$ keyring --help
$ keyring set system username
Password for 'username' in 'system':
$ keyring get system username
password

The command-line functionality is also exposed as an executable package, suitable for invoking from Python like so:

$ python -m keyring --help
$ python -m keyring set system username
Password for 'username' in 'system':
$ python -m keyring get system username
password

https://github.com/jaraco/keyring

So, from Python package keyring point of view, GNOME Keyring is just one of possible back-ends. It can work, for example, also with Windows Credential Locker.

So, what’s the problem? If you on Linux, you have to set-up GNOME Keyring daemon, D-Bus and start D-Bus session. I want to emphasize, you have to install some services like this https://wiki.archlinux.org/index.php/GNOME/Keyring#PAM_method and you have to include some start-up script like this https://lists.alpinelinux.org/~alpine/devel/%3CCAF-%2BOzABh_NPrTZ2oMFUKrsYmSE5obOadKTAth1HU5_OEZUxPQ%40mail.gmail.com%3E

You can install services on the Docker container, there is no problem with it. But when you run your container there is no running services in it, you can’t just have some “start-up script” in Docker container. You can’t rely on int.d / OpenRC / systemd.

Solution will be to use shell script and optionally run it as entrypoint in the docker container.

dbus-launch is utility to start a message bus from a shell script. Quote from its description:

You may specify a program to be run; in this case, dbus-launch will launch a session bus instance, set the appropriate environment variables so the specified program can find the bus, and then execute the specified program, with the specified arguments.

https://www.commandlinux.com/man-page/man1/dbus-launch.1.html

This sounds like the way to go, but unfortunately, this doesn’t work. Quote from documentation:

To start a D-Bus session within a text\(hymode session, do not use dbus-launch. Instead, see dbus-run-session(1).

Ok, let’s look on dbus-run-session:

dbus-run-session — start a process as a new D-Bus session…is used to start a session bus instance of dbus-daemon from a shell script, and start a specified program in that session. The dbus-daemon will run for as long as the program does, after which it will terminate.

https://www.commandlinux.com/man-page/man1/dbus-run-session.1.html

One of the difference between 2 commands is that dbus-run-session “start a process as a new D-Bus session…from a shell script…start a specified program in that session…will run for as long as the program does, after which it will terminate”.

So, in order to use it, we should write our script in CPS-style way, we should call dbus-run-session and pass “the rest of our script” as continuation parameter to this function. Sound, too abstract? Let’s use different perspective.

Quote from keyring documentation:

Using Keyring on headless Linux systems

It is possible to use the SecretService backend on Linux systems without X11 server available (only D-Bus is required). In this case:

* Install the GNOME Keyring daemon.

* Start a D-Bus session, e.g. run dbus-run-session -- sh and run the following commands inside that shell.

* Run gnome-keyring-daemon with --unlock option. The description of that option says:

Read a password from stdin, and use it to unlock the login keyring or create it if the login keyring does not exist.

When that command is started, enter a password into stdin and press Ctrl+D (end of data). After that, the daemon will fork into background (use --foreground option to block).

Now you can use the SecretService backend of Keyring. Remember to run your application in the same D-Bus session as the daemon.

https://keyring.readthedocs.io/en/latest/#using-keyring-on-headless-linux-systems

Let’s go through this quote and try to unpack what it actually says.

keyring is originally designed to work on X Window System (this is another name for X11) — the basic framework for a GUI environment: drawing and moving windows on the display device and interacting with a mouse and keyboard. Docker container are usually without any GUI environment. The good news, that it can run without X Window System, but only with D-Bus.

Note: I will go back to installation (see Setup section below) of D-Bus and GNOME Keyring daemon a bit later.

Quote from above:

Start a D-Bus session, e.g. run dbus-run-session -- sh and run the following commands inside that shell.

I’ve provided a quote from documentation of dbus-run-session. What is happening here, new process is started, new D-Bus session is started and than new shellsh is opened. This shell has access to D-Bus session that was opened a moment before sh was opened, DBUS_SESSION_BUS_ADDRESS environment variable is initialized (more on this below). Note, however, when you will exit this sh shell, D-Bus session will be terminated.

Than I need somehow to run gnome-keyring-daemon --unlock command in new shell sh . The problem is that beside running this command I want to run my Python script.

The quote above illustrate manual process of initialization keyring and I want to provide bash script and even more, I want to run it as entrypoint inside Docker container.

Let’s look on solution:

These scripts are located in /etc/ folder, than they should be run with
--entrypoint=/etc/enter_keyring.sh extra parameter that is passed to docker run command.

This should be done instead standard initialization with int.d / OpenRC / systemd.

Now, I want to explain these 2 scripts line-by-line.

In line 1 we have shebang for bash.

In line 3 we say, that if some command exits with code other than 0, abort script execution.

In lines 5–6 we’re calculated current directory for the bash script.

Here, BASH_SOURCE[0] is the relative path to the script being executed or sourced. Note the difference between BASH_SOURCE[0] and $0: when a script is sourced, BASH_SOURCE[0] will be the path to the script, while $0 will be the path to the bash executable…Now this one-liner should be easy to understand: cd to the directory which contains the current script being executed (or sourced), get the working directory with pwd, and save it in a variable called DIR.

https://medium.com/@Aenon/bash-location-of-current-script-76db7fd2e388

In lines 8-9 we’re invoking dbus-run-session (see above), but we’re passing to it not sh command to open sh shell, but we’re passing

rest_keyring.sh “$@”

script in the same directory.

rest_keyring.sh is bash script that will be invoked as continuation. “$@” means that whatever parameters was passed to the current script, typically this is invoking some python script, they will be forwarded as is, as parameters to rest_keyring.sh.

So, first, dbus-run-session will start new process with new D-Bus session, it will set up environment variables and then it will call rest_keyring.sh with the same parameters that current script was called.

“$@” contains command that you want to exec in docker, for example, python your_python_script.py. If you’re using PyCharm Professional this will contain “an old” entrypoint, that PyCharm Professional put as entrypoint to your Docker container.

Note: It works only if you’re running PyCharm Professional in debug mode.

So, before rest_keyring.sh,was called, new process was started with new D-Bus session. All environment variables was set up. Let’s go through rest_keyring.sh.

In line 1 we have shebang for bash.

In line 3 we say, that if some command exits with code other than 0, abort script execution.

In lines 5–6 we’re calculated current directory for the bash script, see above.

In line 8 we’re saving DBUS_SESSION_BUS_ADDRESS environment variable, that was set up by dbus-run-session to the file /etc/DBUS_SESSION_BUS_ADDRESS for the use by another scripts, that will be described below.

In line 10 we’re killing any gnome-keyring-daemon. Ifrest_keyring.sh runs as part of docker container container entrypoint, this command shouldn’t find any process to kill, but it may also run as part of unlock_keyring.sh (see below) script that is run from bash session, in the latest case this can happen. Anyway, we want to ensure that we have no gnome-keyring-daemon running.

In lines 12–17 the core happens in line: gnome-keyring-daemon --unlock. This command:

…Read a password from stdin, and use it to unlock the login keyring or create it if the login keyring does not exist…When that command is started, enter a password into stdin and press Ctrl+D (end of data). After that, the daemon will fork into background…

https://keyring.readthedocs.io/en/latest/#using-keyring-on-headless-linux-systems

Note, that dbus-run-session have created another process, that is different from PID 1 (process that was started by starting docker container; our entrypoint started to run in PID 1 process).

We don’t want to ask user to enter any password (when we’re starting docker container from PyCharm Proffesional it simply doesn’t work) and we should send somehow Ctrl+D command, this is what you’re seeing in another lines. Also the combined command will export new environment variables that was created by gnome-keyring-daemon --unlock command.

Let’s go over in details.eval is used in combination with a Command Substitution. It is combination of 3 separate commands.

• The first one echo -n “$” passes password $ to the second command. -n is used to disable sending new line as part of echo (otherwise \n will become part of the password).
• The second one gnome-keyring-daemon --unlock is described in detail above as “core command”. After receiving Ctrl+D signal, it creates login keyring (or unlock the login keyring if login keyring do exist). After receiving Ctrl+D signal this commands will prints to stdout something like:

GNOME_KEYRING_CONTROL=/root/.cache/keyring-WA6UU0
SSH_AUTH_SOCK=/root/.cache/keyring-WA6UU0/ssh

This is Unix Socket for communication with Gnome Keyring.

• The third command sed -e ‘s/^/export /‘ prepends ‘export ‘ to the lines above. After receiving Ctrl+D signal they will effectively becomes:

export GNOME_KEYRING_CONTROL=/root/.cache/keyring-WA6UU0
export SSH_AUTH_SOCK=/root/.cache/keyring-WA6UU0/ssh

-e flag is used to o make multiple selections (we want to apply this command on 2 rows).

s means make substitution of start of the line (the caret (^)) to export . This will prepends export to the outputted lines.

Before running line 17, nothing will happen. bash will eval the gnome-keyring-daemon --unlock command, passing $ as password, but nothing will happen. Bash will wait for Ctrl+D signal. This is exactly what we do in line 17. Note, that empty string is passed to stderr, so password is not affected. After receiving Ctrl+D signal login keyring will be created (or unlocked) with password $, the following 2 lines will be printed out to stdout:

export GNOME_KEYRING_CONTROL=/root/.cache/keyring-WA6UU0
export SSH_AUTH_SOCK=/root/.cache/keyring-WA6UU0/ssh

They will set GNOME_KEYRING_CONTROL and SSH_AUTH_SOCK to point to Unix Socket.

In line 20–24 we have optional hook to run some more initialization commands. If you running Python program that expects to have some variables already that has been configured, for example, it expects keyring get system username to return username, you should be able to populate keyring with such values. So, you can put, something like this:

as /etc/enter_init.sh.

If such file is found (lines 20–21), it will be sourced (lines 22–23).

In line 26 we’re calling “continuation” — the “original” entrypoint command, for example, running Python script.

Suppose, we have started our docker container, using --entrypoint=/etc/enter_keyring.sh. Now, we run docker exec ... bash command to open bash session.

For example, we’re in the middle of the debugging of the Python program, that was starting using --entrypoint=/etc/enter_keyring.sh. We want to check/change the state of the keyring. One way to do this, is to open bash session and source/etc/reuse_keyring.sh. This will connect us to the opened D-Bus Session and we can run commands such as keyring get system to check what is stored in keyring.

Let’s look on the script itself.

In line 1 we have shebang for bash.

In line 4 we say, that if some command exits with code other than 0, abort script execution.

In line 6 we’re storing at variable file reference to the file /etc/DBUS_SESSION_BUS_ADDRESS. This is the file where we have stored DBUS_SESSION_BUS_ADDRESS when we opened D-Bus session.

In line 8 we’re checking if file /etc/DBUS_SESSION_BUS_ADDRESS doesn’t exist we’re doing nothing and exiting.

Otherwise, in line 9 we’re fetching the content of /etc/DBUS_SESSION_BUS_ADDRESS to variable a.

In line 10 we’re checking, if a is empty, we’re doing nothing and exiting.

Otherwise, in line 11 we’re setting DBUS_SESSION_BUS_ADDRESS environment variable to the content of /etc/DBUS_SESSION_BUS_ADDRESS file and

In line 11 we’re printing out it.

Note:

1. This script should be sourced and not invoked. If you’re invoking this script changing DBUS_SESSION_BUS_ADDRESS environment variable (of the logged in bash) will fail.
2. This script will do nothing if /etc/DBUS_SESSION_BUS_ADDRESS doesn't exists or empty.

Now, let’s look on another scenario:

We had started docker container without using --entrypoint=/etc/enter_keyring.sh. We want to initialize keyring from the bash. (Maybe we want to run our Python program from the bash).

This is exactly what /etc/unlock_keyring.sh is for. This script should be sourced and not invoked. Lets look on it.

In line 1 we have shebang for bash.

In lines 3–4 we’re calculated current directory for the bash script, see above.

In line 7 we say, that if some command exits with code other than 0, abort script execution.

In line 9 we’re checking if we have dbus-daemon process up and running.

If we do have (this mean we have active D-Bus session):

• At line 10 we’re checking whether we have DBUS_SESSION_BUS_ADDRESS environment variable set. If it is already set, we’re done, we’re exiting the script. Otherwise, At line 11 we’re sourcing reuse_keyring.sh that will reset DBUS_SESSION_BUS_ADDRESS environment variable from another process (via /etc/DBUS_SESSION_BUS_ADDRESS file, see above). At line 13 we’re checking if we don’t have gnome-keyring-daemon process up and running (this shouldn’t happen in practice, theoretically it is possible if, for example, we had killed this process manually, or as part of the script that didn’t rerun it again). /etc/unlock_keyring.sh was sourced, it sources /etc/rest_keyring.sh that starts D-Bus session if needed, starts GNOME keyring and change some environment variables (gnome-keyring-daemon — unlock | sed -e ‘s/^/export /’)). When everything is up and environment variables are set, we can continue in our original (logging?) bash session. We’re done and we’re exiting the script.
• Otherwise, we don’t have active D-Bus session, lines 18–32 will be executed.

Note: This part of code is inspired by the following code (see above why dbus-launch was changed to dbus-run-session):

https://www.commandlinux.com/man-page/man1/dbus-launch.1.html

In line 18–21 we’re checking if DBUS_SESSION_BUS_ADDRESS is non-empty (it shouldn’t happen, but maybe somebody set it up manually?), we will print the warning to stderr and continue the execution (DBUS_SESSION_BUS_ADDRESS will be reset).

In line 23, we’re printing some message to stdout in order to know, that we’re going to initialize D-Bus session and create/unlock GNome keyring (and we will not reuse_keyring.sh).

In line 25 we’re assigning all passed parameters to the script to the param variable. param will typically be empty. If it is non-empty, it typically be invoking python script.

In line 27–29 we’re checking whether param is empty (this is typical case) and if it is the case, we’re assigning bash to param.

In line 31–32 we’re starting D-Bus session in new process, passing as continuation rest_keyring.sh “$param” (param is typically just bash).

Note: For description about rest_keyring.sh see above.

Setup

Now, let’s talk about docker image configuration. It should be something like this:

This Dockefile is based on https://github.com/alex-ber/alpine-anaconda3/blob/master/Dockerfile

Note: alpine-anaconda3 is my dockerfile project for creating Docker Image with latest Anaconda3 packages. It contains “latest” Python 3 packages, see README.md for more details.

All bash script are taken from https://github.com/alex-ber/alpine-anaconda3/

This is main installation part:

#dbus-launch, dbus-run-session
RUN set -ex && \
    apk add --no-cache dbus=1.12.18-r0 dbus-x11=1.12.18-r0 libx11=1.6.12-r0 libxcb=1.14-r1 libxdmcp=1.1.3-r0 \
                           libxcb=1.14-r1 libx11=1.6.12-r0
#libgnome-keyring
RUN set -ex && \
   apk add --no-cache libgnome-keyring=3.12.0-r2 dbus-libs=1.12.18-r0 gnome-keyring=3.36.0-r0 \
           linux-pam=1.3.1-r4 gcr-base=3.36.0-r0 p11-kit=0.23.20-r5 glib=2.64.6-r0 pcre=8.44-r0 \
           libmount=2.35.2-r0 libblkid=2.35.2-r0 libintl=0.20.2-r0 libcap-ng=0.7.10-r1

This part should be added to your Alpine Linux docker container to be able to use GNOME Keyring.

Python (sample) section is just “raw” Python. I want to install keyring only in order to have the ability to test the scripts.

Usage example

Save the Dockerfile above in the directory together with all bash (*.sh) scripts. In the bash type:

docker build . -t test-i

docker run — name test -d test-i

docker exec -it $(docker ps -q -n=1) bash

cd /etc

source unlock_keyring.sh

keyring set https://upload.pypi.org/legacy your-username

Type your password.

See also:

Docker container with Python for ARM64/AMD64