Kai Lüke

Custom port-based BPF firewall loader for systemd services, 2nd part

Sun, 18 Aug 2019 23:51:31 +0000

Update at the end of the post

The first post introduced the IPIngressFilterPath and IPEgressFilterPath feature for the upcoming systemd release 243 to support custom BPF programs as firewall for services. There was a simple example for a minimal BPF program written in C and how to load it for a systemd service. Then there was a standalone program to dynamically control a BPF filter to drop based on packet size.

Now I will show a more realistic use case: a configurable firewall. The new code here can be found in the same repository as the first examples.

Some time ago, a port-based BPF firewall was planned to complement the inbuilt IP-address-based BPF firewall of systemd. It was written in BPF assembly but the PR was not merged mainly because of this fact. The new IPIngressFilterPath and IPEgressFilterPath features in v243 can be used to write a port-based filter as custom BPF program but it’s up to the user to do that.

This blogpost’s program allows you to filter IP packets by ports without writing any BPF code yourself. It not only useful on its own but it also answeres the question how the empty C template from the first post has to be filled out with more than a return 0; which drops all packets.

The road to a working BPF program is quite rough when you get started. There are many BPF examples online but they might use the wrong BPF program type: here we use the cgroup/skb type which works on skb socket buffer pointers, however, the skb data is not an Ethernet frame but an IP packet. Using the old style of helper functions to access packet bytes is also forbidden.

When you find examples of BPF code it is good to answer the basic question where the helper functions come from. There is no unified BPF library and the examples can use different helper definitions (sometimes it’s a simple name change). Here I’ve taken them from the iproute2 repository and not from the kernel repository but there are more to watch out for.

If you just want to check some ports or parse the HTTP header, you first need to parse the IP packet which can be a daunting task because even if you don’t need to handle Ethernet frames you still have to skip through the IPv4/v6 headers with varying length. The best is to reuse the kernel structs by casting the addresses of packet offsets to them. It’s not hard but unfamiliar if you haven’t dealt with kernel internals and IP headers before. Not to mention the tricks needed to pass the BPF verifier when trying to load the compiled bytecode into the kernel.

The new BPF program parses the packet headers and drops or forwards the packets based on the metadata extracted from the packet headers. The forward/drop configuration is static for now, meaning that you need to reload the BPF program to change the configuration. The forward/drop filter expression can involve destination and source ports for TCP and UDP, the IP version (being IPv4 or IPv6) and the protocol types ICMP, TCP, or UDP.

The filter expression is a C expression over the boolean variables udp, tcp, icmp, ip, and ipv6 denoting the packet type and over the the integers dst_port and src_port for the UDP/TCP ports. The filter is passed as macro definition to the C BPF program. Valid filters examples are FILTER='icmp || (udp && dst_port == 53) || (tcp && dst_port == 80)' or FILTER='!udp || dst_port == 53'.

The following C BPF program can be compiled through (assuming the environment variable FILTER is set):

$ clang -O2 -Wall -target bpf -c port-firewall.c -o port-firewall.o -D "FILTER=${FILTER}"

You can find the source with a makefile to compile and load the BPF program here in the port-firewall directory.

/* Copyright 2019 Kai Lüke <kailueke@riseup.net>
 * SPDX-License-Identifier: GPL-2.0
 *
 * Minimal configurable packet filter, parses IP/IPv6 packets, ICMP, UDP ports,
 * and TCP ports. The forward rule is a C expression passed as FILTER variable
 * to the compiler with -D. The expression can use the boolean variables
 * [udp, tcp, icmp, ip, ipv6] and the integers [dst_port, src_port].
 * If the expression evaluates to 0 (false), the packet will be dropped.
 */
#include <linux/bpf.h>
#include "bpf_api.h"
#include <linux/in.h>
#include <linux/if.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <linux/if_tunnel.h>
#include <linux/icmp.h>
#include <linux/tcp.h>
#include <linux/udp.h>
#include <sys/socket.h>
#include <linux/if_packet.h>

#ifndef __section
# define __section(NAME)                  \
   __attribute__((section(NAME), used))
#endif

/* cgroup/skb BPF prog */
__section("filter")
int port_firewall(struct __sk_buff *skb) {
  __u8 udp = 0, tcp = 0, icmp = 0, ip = 0, ipv6 = 0;
  __u16 dst_port = 0;
  __u16 src_port = 0;

  void *data = (void *)(long)skb->data;
  void *data_end = (void *)(long)skb->data_end;

  ip = skb->protocol == htons(ETH_P_IP);
  ipv6 = skb->protocol == htons(ETH_P_IPV6);

  if (ip) {
    if (data + sizeof(struct iphdr) > data_end) { return 0; }
    struct iphdr *ip = data;
    /* IP fragmentation does not need to be handled here for cgroup skbs */
    icmp = ip->protocol == IPPROTO_ICMP;
    tcp = ip->protocol == IPPROTO_TCP;
    udp = ip->protocol == IPPROTO_UDP;
    if (udp || tcp) {
      __u8 *ihlandversion = data;
      __u8 ihlen = (*ihlandversion & 0xf) * 4;
      if (data + ihlen + sizeof(struct tcphdr) > data_end) { return 0; }
      struct tcphdr *tcp = data + ihlen;
      src_port = ntohs(tcp->source);
      dst_port = ntohs(tcp->dest);
    }
  } else if (ipv6) {
    struct ipv6hdr *ipv6 = data;
    __u8 ihlen = sizeof(struct ipv6hdr);
    if (((void *) ipv6) + ihlen > data_end) { return 0; }
    __u8 proto = ipv6->nexthdr;
    #pragma unroll
    for (int i = 0; i < 8; i++) { /* max 8 extension headers */
      icmp = proto == IPPROTO_ICMPV6;
      tcp = proto == IPPROTO_TCP;
      udp = proto == IPPROTO_UDP;
      if (udp || tcp) {
        if (((void *) ipv6) + ihlen + sizeof(struct tcphdr) > data_end) { return 0; }
        struct tcphdr *tcp = ((void *) ipv6) + ihlen;
        src_port = ntohs(tcp->source);
        dst_port = ntohs(tcp->dest);
      }
      if (icmp || udp || tcp) {
        break;
      }
      if (proto == IPPROTO_FRAGMENT || proto == IPPROTO_HOPOPTS ||
          proto == IPPROTO_ROUTING || proto == IPPROTO_AH || proto == IPPROTO_DSTOPTS) {
        if (((void *) ipv6) + ihlen + 2 > data_end) { return 0; }
        ipv6 = ((void *) ipv6) + ihlen;
        proto = *((__u8 *) ipv6);
        if (proto == IPPROTO_FRAGMENT) {
          ihlen = 8;
        } else {
          ihlen = *(((__u8 *) ipv6) + 1) + 8;
        }
        if (((void *) ipv6) + ihlen > data_end) { return 0; }
      } else {
        break;
      }
    }
  }

  if (FILTER) {
    return 1; /* 1 = forward */
  }
  return 0; /* 0 = drop */
}

char __license[] __section("license") = "GPL";

The checks with > data_end are needed to ensure that only valid memory is accessed. The loop unroll is another thing done for the BPF verifier because support for loops is not available yet. I used the calculation ihlandversion & 0xf for the IPv4 header length because I didn’t test if using the kernel struct union byte works on both big and little endian systems (I guess it should work). If you have improvements please share them on the GitHub repository.

The makefile requires to pass the filter when the program is built:

$ make FILTER='…'
$ # now we can load to /sys/fs/bpf/port-firewall:
$ make load

After compilation the makefile can load the program via sudo $(which bpftool) prog load port-firewall.o /sys/fs/bpf/port-firewall type cgroup/skb.

As mentioned, systemd v243 has the options IP(Ingress|Egress)FilterPath= for use in service files or systemd-run and systemd-nspawn. Look up how to use them in the first post or in the systemd documentation. It would be fun to use systemd’s service file template mechanism for the filter-loading units to instantiate on-demand compilation and loading of filters based on the forward/drop expression. If you write a loading service for a single filter expression, look at the first post and the bpf-program-only/ folder in the repository on how to do that.

For quick testing when systemd v243 is not available you can use systemd-run to spawn a shell in a new cgroup: either as system service with sudo systemd-run --scope -S or as user service with systemd-run --user --scope -S (where -S can be replaced with a concrete binary instead of starting a shell). This will print out the unit name which is also the name of the cgroup. The full cgroup path for the system service shell is /sys/fs/cgroup/unified/system.slice/NAME. For the user service shell the path is /sys/fs/cgroup/unified/user.slice/user-1000.slice/user@1000.service/NAME depening on your UID not being 1000.

Then attach the BPF program to the cgroup: sudo $(which bpftool) cgroup attach /sys/fs/cgroup/unified/user.slice/user-1000.slice/user@1000.service/run-rfaa93ac79de2482d8ef1870fd6b508cd.scope egress pinned /sys/fs/bpf/port-firewall multi. You can either choose ingress or egress to filter incoming or outgoing packets. You can load the same filter for both ingress and egrees and you can load multiple different filters per ingress/egress (also true when used through the systemd v243 option). As mentioned in the sidenote of the first post, if you turn on IPAccounting in systemd-run you need to turn on Delegate as well to allow multiple BPF programs.

For your convenience I’ve again included example systemd service files, bpf-make.service to configure and load the BPF program, and my-filtered-ping.service which uses the loading service. They differ a bit from the ones in bpf-program-only folder from the first post because they use the makefile to first compile the program with a configuration.

Here is bpf-make.service which will be used as helper service for our ping service:

[Unit]
Description=BPF port-firewall load service

[Service]
Type=oneshot
RemainAfterExit=yes
# If bpftool is not installed system-wide use: Environment="PATH=/bin:/usr/bin:/path/to/bpftool-folder"
Environment='FILTER=icmp || (udp && dst_port == 53) || (tcp && dst_port == 80)'
ExecStart=/usr/bin/make -C /path/to/repo/bpf-cgroup-filter/port-firewall ; make -C /path/to/repo/bpf-cgroup-filter/port-firewall load
ExecStop=rm /sys/fs/bpf/port-firewall
LimitMEMLOCK=infinity

Here is my-filtered-ping.service that uses the loaded filter:

[Unit]
Description=my egress-filtered ping service
Requires=bpf-make.service
After=bpf-make.service

[Service]
ExecStart=ping 127.0.0.1
IPEgressFilterPath=/sys/fs/bpf/port-firewall
# If you don't have systemd v243 you can use this instead of the above line:
# ExecStartPre=/path/to/bpftool cgroup attach /sys/fs/cgroup/unified/system.slice/my-ping.service egress pinned /sys/fs/bpf/port-firewall multi

Either use the IPEgressFilterPath… line if you have systemd v243 or the ExecStartPre… line as workaround with bpftool. The ping is expected to work because we filtered egress but allowed ICMP packets, UDP on the DNS port, and TCP on port 80 (HTTP but not HTTPS).

That’s it for now. I hope the systemd v243 release will be out soon and people can use this small program here to add port filters to their services or as inspiration to write their own BPF filters. In my head I have ideas for a bandwidth throttling filter and a PCAP packet dumping filter.

UPDATE: Systemd unit templates

The above version of the loader unit file has to be copied for every filter. Through a unit template file we can avoid this (required an update in the makefile):

[Unit]
Description=BPF port-firewall load service template for filter: %I

[Service]
Type=oneshot
RemainAfterExit=yes
# If bpftool is not installed system-wide use: Environment="PATH=/bin:/usr/bin:/path/to/bpftool-folder"
Environment='FILTER=%I'
Environment='BPFNAME=%i'
ExecStart=/usr/bin/make -C /path/to/repo/bpf-cgroup-filter/port-firewall
ExecStop=/usr/bin/make -C /path/to/repo/bpf-cgroup-filter/port-firewall remove
LimitMEMLOCK=infinity

Templates unit behave like regular units but carry the parameter in the unit name, e.g., "bpf-firewall@icmp || (udp && dst_port == 53) || (tcp && dst_port == 80).service". The parameter ends up being encoded with systemd-escape and the makefile will use this as program name in the BPF filesystem.

We could now use this in the Requires and After sections and the encoded filter string in IPIngressFilterPath/IPEgressFilterPath. But that means that every change in the filter needs three changes in the file.

Here is a hacky solution, making the final service a template unit itself so that the filter has to be specified as part of the service name:

[Unit]
Description=my egress-filtered ping service template
# To avoid specifying the FILTER here twice and below again,
# this service file is a template and the FILTER has to be
# passed as argument (referenced with %i) when instanciating
# the service via `systemctl start "service-with-filter@FILTER.service"`
Requires=bpf-firewall@%i.service
After=bpf-firewall@%i.service
# The alternative is to not use a template file and include the argument directly
# here as bpf-firewall@ESCAPED.service with ESCAPED being the output
# of `systemd-escape "FILTER"`.
# Then you can make this file here a regular service file without the @.

[Service]
ExecStart=ping 127.0.0.1
IPEgressFilterPath=/sys/fs/bpf/%i

# If you don't have systemd v243 you can use this instead of the above line:
# ExecStartPre=/path/to/bpftool cgroup attach /sys/fs/cgroup/unified/system.slice/system-service\x5cx2dwith\x5cx2dfilter.slice/%n egress pinned /sys/fs/bpf/%i multi
# Cannot use %p here but have to use 'my\x5cx2dping\x5cx2dwith\x5cx2dfilter' (encoded twice with systemd-escape) because the cgroup fs slice path name still has the escaping and if we use the escaping here once it is reverted once and thus removed when the unit is loaded.

# Again, if this file is not a template, instead of %i use
# IPEgressFilterPath=/sys/fs/bpf/ESCAPED
# Without systemd v243 it would become
# ExecStartPre=/path/to/bpftool cgroup attach /sys/fs/cgroup/unified/system.slice/%n egress pinned /sys/fs/bpf/ESCAPEDTWICE multi
# with ESCAPEDTWICE being the output of `systemd-escape "ESCAPED"`.

With that file for our final service to start, the filter has to be specified as part of the service: systemctl start "service-with-filter@icmp || (udp && dst_port == 53) || (tcp && dst_port == 80).service". Not beautiful but it does its job. Ideally I would store the filter in a BPF map so that it can be set after BPF program loading, allowing it to be a simple ExecStartPre line in the service file.

Custom BPF firewalls for systemd services

Sat, 29 Jun 2019 22:38:31 +0000

The upcoming systemd release 243 will allow to specify custom BPF programs as firewall for systemd services. I implemented this as coding task from my interview at Kinvolk.

BPF programs are small bytecode programs the Linux kernel can run in its network stack or attached to syscalls and other events. BPF is sometimes called eBPF because it extends the original Berkley Packet Filter (BPF) used to pre-filter packets for RAW sockets (this classic BPF is now called cBPF). Read more about BPF here.

The usage of BPF in this post here is limited to programs attached as filters for IP packets of all sockets in a cgroup (control group). Cgroups are actually hierarchical which also means that filters of the parent cgroup will also apply. In addition, multiple BPF programs per cgroup may be attached. The filtering happens after IP reassembly/before fragmentation.

Systemd puts the service (unit) process in a new cgroup and any children processes will also stay there. In 2017 systemd got the ability to have per-unit IP accounting and filtering based on IP addresses which is done with exactly those cgroup socket filters. There was a plan to add port filtering as well but the way systemd ships BPF code meant that it had to be done in BPF assembly.

The task I took up was planned here and proposed to offload the BPF program creation to the user instead of having it done by systemd. This allows to have port filters or even HTTP-aware filters supplied by an external program by pinning the BPF programs to /sys/fs/bpf/….

My commit adds two new properties for systemd units. IPIngressFilterPath and IPEgressFilterPath both take a path to a pinned BPF program. While the properties take a single path, you can specify them multiple times to attach more than one filter. They are combine as queue. An empty assignment to the property resets all previous filters. Of course we don’t have to drop any packets and can also use this machinery to monitor or modify packets.

The filters can be specified in service unit files, or with systemd-run or systemd-nspawn. I will start with giving an example that uses systemd-run, first with a simple filter compiled from C, then with an interactive MTU filter. You can find the examples in this repository as well.

Simple dropping filter example

The simplest way of writing a BPF program is using C and compiling with clang to BPF bytecode (but there are some limitations and workarounds to get to know). Here is a source file with a filter that just drops any packet:

/* cgroup/skb BPF prog */
#include <linux/bpf.h>

#ifndef __section
# define __section(NAME)                  \
   __attribute__((section(NAME), used))
#endif

__section("filter")
int cgroup_socket_drop(struct __sk_buff *skb)
{
    /* analyze skb content here */
    return 0; /* 0 = drop, 1 = forward */
}

char __license[] __section("license") = "GPL";

This can be compiled to an object file:

$ clang -O2 -Wall -target bpf -c cgroup-sock-drop.c -o cgroup-sock-drop.o

To load and pin it to the BPF filesystem we will use the bpftool binary. Install bpftool as package in Fedora (sudo dnf install bpftool) or compile and copy it to your PATH from a kernel source directory (cd ~/linux-source-x.xx/tools/bpf/bpftool ; make bpftool ; cp bpftool ~/.local/bin/).

Then we can load our small filter:

$ sudo `which bpftool` prog load cgroup-sock-drop.o /sys/fs/bpf/cgroup-sock-drop type cgroup/skb

Unloading is simply done with rm /sys/fs/bpf/cgroup-sock-drop and is needed before an updated version can be pinned to that location.

Using a filter with systemd-run/nspawn

Given you read this when the future systemd 243 release is installed, we can now use our BPF program as ingress filter for a temporary ping service:

$ sudo systemd-run -p IPIngressFilterPath=/sys/fs/bpf/cgroup-sock-drop --scope ping 127.0.0.1
Running scope as unit: run-re62ba1c….scope
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
^C # cancel since it will not get responses
--- 127.0.0.1 ping statistics ---
8 packets transmitted, 0 received, 100% packet loss, time 186ms

We could specify more than one ingress filter or egress filter but since the first one already drops everything there would not be any difference. The syntax for systemd-nspawn is similar and allows to use our BPF filter as firewall for a whole container: systemd-nspawn --property=IPIngressFilterPath=/sys/fs/bpf/cgroup-sock-drop ….

Using service files

Since service files will probably be run at boot time, we need a service in /etc/systemd/system/cgroup-sock-drop-filter.service that loads and pins our filter (change the /path/to parts):

[Unit]
Description=cgroup socket drop filter

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/path/to/bpftool prog load /path/to/cgroup-sock-drop.o /sys/fs/bpf/cgroup-sock-drop-filter type cgroup/skb
ExecStop=rm /sys/fs/bpf/cgroup-sock-drop-filter
LimitMEMLOCK=infinity

Note the last line without the loading will fail. We can use this filter now in multiple services, both as ingress or egress filter. Here comes our ping example again as /etc/systemd/system/my-ping.service:

[Unit]
Description=my ping service
Requires=cgroup-sock-drop-filter.service
After=cgroup-sock-drop-filter.service

[Service]
ExecStart=ping 127.0.0.1
IPIngressFilterPath=/sys/fs/bpf/cgroup-sock-drop-filter

Of course you need some more other lines to run it at startup as usual (e.g., After=network.target and an [Install] section with WantedBy=multi-user.target). Anyway we now run systemctl start my-ping.service and see its output when we run systemctl status my-ping.service. The output should be the same as previously.

Interactive MTU filter example

BPF programs and userspace programs can communicate through BPF maps. This makes it possible to write a BPF filter that drops packets, e.g., larger than a certain size stored in a BPF map. Then a userspace program can change this value dynamically. In the repository linked above you can find a small load_and_control_filter tool doing that in the standalone folder. It simulates the behavior of a small MTU (Maximum Transfer Unit) on a network path or packet drops when the MTU is changed to 0.

The tool loads a BPF cgroup ingress/egress filter bytecode that filters based on the packet size. It then pins the BPF filter to a given location in /sys/fs/bpf/. Through the +/- keys the MTU can be changed interactively (actually changes the values in the BPF map). Optionally the initial MTU value can be specified on startup. The program can also attach the BPF filter to a cgroup by specifying the cgroup by its path. The BPF filter stays loaded when the program exits and has to be deleted manually with rm.

In contrast to the first example, it does not use a BPF compiler but uses hardcoded BPF assembly instructions to include the BPF code in the final program. This is not very accessible for hacking on it but for me it was interesting to see how BPF instructions work and what needs to be done to comply with the verifier.

In one terminal we can run the interactive filter:

$ sudo ./load_and_control_filter -m 100 -t ingress /sys/fs/bpf/ingressfilter
cgroup dropped 0 packets, forwarded 0 packets, MTU is 100 bytes (Press +/- to change)
… # keeps running

This loads the BPF filter to /sys/fs/bpf/ingressfilter which we can use for a systemd service the same way as the first dropping filter.

In a second terminal we will run ping again as root in a temporary systemd scope and specify our filter as IPIngressFilterPath:

$ sudo systemd-run -p IPIngressFilterPath=/sys/fs/bpf/ingressfilter --scope ping 127.0.0.1
Running scope as unit: run-….scope
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.086 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.069 ms
…

When we switch back to the first terminal and press -, the new MTU is 50 bytes and we can see the dropped packet count increase. In the ping terminal we will see no new responses because they are all dropped.

Attaching a filter to a cgroup without systemd 243

The load_and_control_filter program can be told to attach the filter to a cgroup of a systemd service. This just needs the path of the cgroup which is derived by the unit name.

Sidenote: Systemd uses a BPF filter for its IP accounting and firewalling based on IP addresses. If such a filter is present but no others, the flag to allow multiple BPF filters for a cgroup is missing. As workaround when, e.g., IP accounting is enabled, we can tell systemd that the cgroup management is done by externally. This means that systemd will use the flag to allow multiple BPF filters instead of loading the IP accounting BPF filter without this flag.

Using systemd-run again for a temporary service/scope:

$ sudo systemd-run -p IPAccounting=yes -p Delegate=yes --scope ping 127.0.0.1
Running scope as unit: run-r9f31b3947f4c4a11a24babf5517fe025.scope
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.086 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.069 ms
…

We can see the scope name in the first output line. This is also the last part of the cgroup path we have to use as argument in order to attach the filter to the cgroup.

$ sudo ./load_and_control_filter -m 100 -c /sys/fs/cgroup/unified/system.slice/run-r9f31b3947f4c4a11a24babf5517fe025.scope -t ingress /sys/fs/bpf/myfilter
cgroup dropped 0 packets, forwarded 4 packets, MTU is 100 bytes (Press +/- to change)
… # keeps running and increases the forward count

Now we can hit - to reduce the MTU and observe the packet drop count increasing while no ping responses can be seen. We could also start an additional egress filter to observe what happens to ping when outgoing packets are dropped. This is actually propagated through the sendmsg syscall returning Operation not permitted.

I hope that someone will use the new systemd properties for more interesting filters than the both above. The interactive MTU filter may be useful to try out if PLPMTU kicks in for a TCP connection with echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing and appropriate values in …/tcp_base_mss and …/tcp_probe_*.

I also hope that this makes BPF filters more accessible for users. I imagine that a filter for, e.g., HTTP paths would be loaded and combined with another one for ports, both provided by systemd services in a reusable fashion (templates with @?). Maybe a filter has enough generality or contextual information to be useful for multiple services so that they all share a single instance. Maybe it also turns out to be useful for user services (test that with systemd-run --user …). You can reach out to me via mail or GitHub issues in the repository for the above examples.

The feature is actually quite simple and I don’t blog often but a feature that is not explained well may stay unused. I’m thankful for the idea to work on this, provided to me in my interview at Kinvolk. My experience in GNOME and the research at KAIST’s Advanced Networking Lab with Prof. Sue Moon have probably helped me to get this done. The last word of thanks goes to Lennart Poettering for the review and guidance to get the PR merged.

GNOME Disks Development News

Wed, 27 Jun 2018 01:04:31 +0000

The most visible change for GNOME Disks in 3.28 was the new format dialog. Unfortunately the redesign of the partition view has not been tackled in the last development cycle.

A contribution from Andrea Azzarone improves the experience with snap applications because GNOME Disks will now hide all images mounted with the x-gdu.hide option, e.g. sudo mount special.img /mnt/test/ -o loop,x-gdu.hide

Also several bugs are fixed in 3.28.3, so I recommend distributions to update. A Flatpak release is not yet available and due to some minor issues regarding the integration with the host system I don’t know yet if this really makes much sense.

For 3.30 there is support for opening VeraCrypt images, which was contributed by segfault and the Tails project. This needs a UDisks version which is not yet available and requires the creation of the file /etc/udisks2/tcrypt.conf if not shipped by the distribution.

Finally, the gnome-disk-utility repository has been moved to GNOME’s GitLab. The repository also contains the code for gnome-disk-image-mounter and the gsd-disk-utility-notify service for S.M.A.R.T. events. I hope this will attract some more people to work on GNOME Disks as I am currently not able to do much for it.

Interaction Between the User and Kernel Space in Linux

Sun, 21 Jan 2018 13:52:12 +0000

Seminar Paper June 2017, Interaction Between the User and Kernel Space in Linux, Kai Lüke

Abstract:

System calls based on context switches from user to kernel space are the established concept for interaction in operating systems. On top of them the Linux kernel offers various paradigms for communication and management of resources and tasks. The principles and basic workings of system calls, interrupts, virtual system calls, special purpose virtual filesystems, process signals, shared memory, pipes, Unix or IP sockets and other IPC methods like the POSIX or System V message queue and Netlink are are explained and related to each other in their differences. Because Linux is not a puristic project but home for many different concepts, only a mere overview is presented here with focus on system calls.

Download paper: Interaction Between the User and Kernel Space in Linux, Kai Lüke

Converted to HTML with pdf2htmlEX: Interaction Between the User and Kernel Space in Linux, Kai Lüke

Last Project Phase and 3.26 Features

Tue, 22 Aug 2017 10:32:32 +0000

Repair and resize is available in the recent 3.25 release and needs at least UDisks 2.7.2. Currently Ext4, XFS and FAT are supported through libblockdev and I hope to extend this list with NTFS soon. There were some race conditions when a resized partition is detected by the kernel again and also the FAT support through libparted is still a bit shaky. Showing the proportion of used disk space in the slider was the last appearance change for the resize dialog. I’ve written some retrospective words about the project at the end of its wiki page – thank to all for this learning experience!

The new format dialog did not get merged yet and will come to master after the freeze. Not yet implemented are the mockups for the whole UI where the partition list is shown. Jimmy Scionti and others in #gnome-design worked on my changes to Allan’s original mockup and the direction seems to stabilize now.

It was nice to visit GUADEC and meet people in person and discuss various things in the days after as well. There are too many areas where I also would like to do something but rightnow the time is short, and the list of plans for Disks is also growing…

Development News

Both 3.25.90 and 3.25.91 have been released and I think that 3.26 will be a good improvement compared to 3.24. Please report issues you experience.

Migrated comments:

Stu says:
22nd August 2017 at 17:31
Fantastic 🙂
One thing I think no disk utilities seem to do is differentiate between say USB sticks and SD cards and the internal disk.
Since it takes exactly the same steps to format both, I’m always worried I might not be paying attention and accidentally click through and kill the computers disk, when I meant to repartition yet another SD card.
I guess this is a question for design though.
Kai says:
23rd August 2017 at 13:13
When starting Disks there are different icons on the left side for the devices and also their names should help to distinguish.The format dialog asks for confirmation where you will see the device described again.

GNOME Disks: Integrate Resize and Repair

Tue, 25 Jul 2017 19:46:30 +0000

The basic prototype patches let GNOME Disks resize partitions and their filesystems, check filesystems and repair them – at least when built against UDisks master where the actual work is performed. It needs some fixes on how jobs and UI states correspond but here a glimpse on how the resize looks like.

The current pending mockups for the whole UI will be discussed in person at GUADEC. Where to show the result of the filesystem check or repair and where to show the jobs are some of the questions.

In the implementation some problems pop up as usual, this time with some impact on the robustness but hopefully nothing again at the level of dealing with libparted geometry configuration. The prototype needs some polishing as well, and in the underlaying layers I would like to add NTFS support and improve the FAT resize support. Finally it would be nice if the UDisks jobs indicate the progress which is not supported in libblockdev yet.

Resizing other block devices than partitions is also left to do (e.g. LUKS, backing loop file). Offering a filesystem resize where the block device can’t be fitted is a strange experience because it’s in general impossible to tell up to which size a filesystem occupies space in this block, meaning the result would look the same except for a difference in free space. Thus for the beginning only filesystems in partitions are resizable.

Development News

Recently 3.25.4 was released and the last changes improved the build experience a lot – Iñigo Martínez gave a meson port as surprise gift. Let’s see whether it’s enough time for the ongoing work to land on master for 3.26.

Migrated comments:

Stu says:
26th July 2017 at 02:03
This is a great start, definitely looks much more useful for managing removable devices like USB sticks.
Debarshi Ray says:
28th July 2017 at 23:17
Exciting! Thanks for working on GNOME Disks.
biosin says:
29th July 2017 at 00:11
Appreciate the work. Will it also be possible to move partitions and queue multiple operations like in gparted? Like resize and move in one step.
Kai says:
29th July 2017 at 01:31
Moving is currently not planned, but it would be possible in a similar way the save/restore is working. Queueing up would not be needed if the move can also be directly specified during the resize. We should consider this for the future!

UDisks Repair and Resize API

Tue, 27 Jun 2017 08:54:05 +0000

Almost all actions in GNOME Disks rely on the UDisks system service. That way the authorization policy is handled by PolKit and Disks does not have to spawn any root processes. Because format and mount jobs are registered there to the device objects it is possible to initiate them from command line via gio mount, udisksctl or another D-Bus client such that Disks can still show these ongoing operations in the UI.

There are cases where D-Bus services are not possible (e.g. in an OS installer) and therefore libblockdev is a new library with similar functionality. The last UDisks release offloads many tasks to it instead of spawning utility binaries itself. In libblockdev the preference is to reuse libraries like libparted if possible and otherwise spawn utility binaries.

To have resize and repair support in Disks I had to introduce querying for installed support of an action and the action itself in UDisks and thus in libblockdev. The advantages are only small changes in Disks and Cockpit will also benefit. Now there are PRs pending for a generic repair and check function, a partition resize function and query functions for repair, resize and consistency check support of filesystem types in libblockdev. Then the PRs pending for UDisks can make use of them to expose the following API methos:

org.freedesktop.UDisks2.Manager:
CanFormat(fstype) -> (available, required_utility)
CanResize(fstype) -> (available, mode, required_utility)
CanCheck(fstype) -> (available, required_utility)
CanRepair(fstype) -> (available, required_utility)

org.freedesktop.UDisks2.Partition:
Resize(size, options)

org.freedesktop.UDisks2.Filesystem:
Resize(size, options)
Check(options) -> consistent
Repair(options) -> repaired

The resize mode flags indicate if online or offline shrinking/growing is supported. The size argument for partitions should guarantee that content of this size will fit in and alignment might only make it bigger. The progress percentage property of the running job is not yet set because this needs parsing stdout in libblockdev, so just a spinner in Disks for now. It’s possible to test them from d-feet, e.g. to see if the F2FS format is available and what utility is missing if not:

On the command line it would be gdbus, _mdbus2_ or dbus-send or the D-Bus library of your choice (e.g. in Python3 from gi.repository import Gio). Here assuming an almost empty unmounted ext4 partition 1 on a loop device:

$ # repair first (the option dict {} is unused)
$ gdbus call --system --dest org.freedesktop.UDisks2 \
 --object-path /org/freedesktop/UDisks2/block_devices/loop0p1 \
 --method org.freedesktop.UDisks2.Filesystem.Repair {}
(true,)

$ # shrink filesystem to 50 MB
$ gdbus call --system --dest org.freedesktop.UDisks2 \
 --object-path /org/freedesktop/UDisks2/block_devices/loop0p1 \
 --method org.freedesktop.UDisks2.Filesystem.Resize 50000000 {}
()

$ # shrink partition to 50 MB
$ gdbus call --system --dest org.freedesktop.UDisks2 \
 --object-path /org/freedesktop/UDisks2/block_devices/loop0p1 \
 --method org.freedesktop.UDisks2.Partition.Resize 50000000 {}
()

GNOME Disks Development News

After Disks 3.25.2 was released there were only small changes in master. The standby menu entry was fixed to be usable again after the action took effect. Mitchell took care of large file support on 32-bit systems, an interesting topic. The biggest current change is the new format dialog. The new UDisks API must be integrated now in the UI. It would be nice to get NTFS (and progress) support in libblockdev and finally the Disks UI mockups done as well.

Migrated comments:

Emmanuele Bassi says:
27th June 2017 at 13:50 Uhr
I strongly urge people working on udisk to actually fix their build system, and to test all permutations.
I had to pin udisks in GNOME Continuous because of libblockdev and udisk constantly breaking and being tragically bad with non-srcdir builds (which tells me nobody runs make distcheck either); jhbuild users are pinned to their system version of udisk for the same reason.
This means that gnome-disk-utility changes won’t be buildable by Continuous and jhbuild.
„It builds on my laptop and when I make an RPM package for RHEL/Fedora“ is not a sustainable development model.
Sébastien Wilmet says:
27th June 2017 at 20:39
It’s great that someone works again on gnome-disk-utility! I hope that after your studies you’ll be hired by a company working on GNOME 😉

An Observation in UI Design

Sun, 11 Jun 2017 22:23:09 +0000

Reading the GNOME Human Interface Guidelines gives a good idea on how to arrange elements and reduce complexity. The HIGs also emphasize on having a clear goal which helps in deciding which elements need to be arranged at all. But I did not grasp the wideness of being purpose-driven for this goal of the application which might then mean to abstract from technical details on the way. So now I try to explain this observation here.

I think there are two opposite paradigms one can follow in UI design, and thus what users can expect from UIs. This is not a thorough research on the topic, so you can leave a comment if you have insights to share on whether this view is mistaken. Maybe you will find your own examples during reading if you try to think about image, audio and video formats and their parameters, or network connection and encryption stuff in UIs.

The first paradigm is to make features accessible in the UI. This could just mean showing technical choices and configuration values coming from an API or command line tools. The UI serves as big toolbox with few limits, allowing to be creative and meet various use cases. But from the viewpoint of a newbie it is a big frustration to use these kind of applications because the options don’t have a meaning. Reaching the point where all implications of the parameters are understood is not possible in a few minutes. The whole research task for appropriate parameters has been offloaded to the user.

Therefore, the other paradigm comes into play and thinks from the main purpose of the UI and what elements it needs to reach the goal. That’s what I expect in GNOME. The technical details go to the background and may not be found as UI elements anymore. This needs clever ways to find optimal parameters, or studies of the technical and social/psychological conditions to reduce the workload for the user. First this looks like hiding ‘features’ and configuration options (i.e. technical details) but it might be good to take a step back and look at the state of IT. Is it really necessary that everyone fiddles around with the available technical options? Can the decision for a configuration value be automated or evaluated? And how did we get into the situation of having so many competing ‘standards’ and a rag rug of solutions for the different layers involved and all this strange legacy stuff needed for compatibility.

Maybe we first need to work on a algorithms, libraries and single standards in order to be able to reduce UI complexity. But it’s not always possible to find unique technical solutions for the whole planet and we have to expose some of this strange acronyms and abbreviations in the UI and hope that users have an idea of their meaning. The result is a mixed approach of both paradigms where necessary.

Example: The format dialog in GNOME Disks

In the specific case of the format dialog for pen drives or hard disks you could simply decide to make features of the command line tools available in the UI. This would involve to expose the choice of the filesystem type and if it should be contained in an ecryption layer. Not speaking of overwhelming non-default parameters this toolbox is already hard to understand. For the new design of the format dialog I first proposed this toolbox paradigm approach (even if preference of the tools is indicated and not all are directly visible):

Assessing every option for the given use case and conditions is a lot of work and sometimes a final judgement can’t be made concerning issues with inter-OS compatibility, snapshots, Linux file permissions, encryption and performance (storage type, in-kernel drivers). With the Opus audio codec joint efforts came finally to a good general solution, maybe too late. But for filesystems it’s much more complex and the operating systems also have significant differences.

So if we want to use the result-oriented paradigm, we already see the problem that the UI has to guide the user through these OS compatibility issues. But this still means abstracting from selecting a concrete filesystem type and focusing more on use cases which are tied to filesystem choices (here still visible in brackets). The UI does not have to differ too much but important is the base idea. Here we see Allan’s design:

Ext4, NTFS and FAT are the current default filesystems in Disks but maybe exFAT is already more relevant and could e.g. replace both NTFS and FAT or even cover a distinct use case – that needs further research.

But also for Linux-only use cases there are other filesystems than Ext4 which could replace it as default choice or at least cover special demands like snapshots or being suited for flash storage. Listing all these use cases would almost have the same effect like listing the filesystems with explanation, and that is what should be avoided for a common formating action of an external hard drive to back up some files.

A pragmatic middle path would e.g. as above show swap partitions and XFS for large amounts of data. Yet I would argue that once people set up swap partitions or e.g. Btrfs volumes they can just handle the toolbox paradigm during that step. So as remembered from install wizards, the format dialog could do a paradigm shift and expose the toolbox if a custom filesystem setup is chosen instead of the defaults:

I think that this flexibility is needed because I don’t see a single filesystem to emerge soon which fits all the demands. But I would highly appreciate if people do not have to think about filesystems anymore because operating systems switched to let’s say Btrfs in collaboration and even your digital camera can handle it.

With this fallback available there is less pressure in finding optimal choices for the major use cases. It can be observed if they need to be adjusted instead of campaigning for agreement on e.g. a single Linux-only and also a single inter-OS filesystem. Still it would be nice to find out how user interaction can be reduced and less options presented by e.g. user surveys or applying automatic constraints based on the context.

Migrated comments:

Phil says:
12th June 2017 at 06:25
I mostly agree with what is written, especially with the two ways to approach UI design. I have used both, going a use case driven approach for end user oriented applications and directly exposing technical all technical details mostly in internal tools for development.
Just one note on your example: I think formatting a disk should be easy even for non-technical users, and the choices presented in the second mockup make sense. But just as you wrote yourself I am convinced that this special case warrants an advanced mode where one can select any filesystem. This topic just is too
Alexandre Franke says:
12th June 2017 at 19:39
Convergence towards a single filesystem won’t ever happen and even if it did you’d still have to deal with legacy hardware that is not compatible with it.
Hugo says:
13th June 2017 at 22:55
I totally agree with you
CJ says:
13th June 2017 at 01:11
Good point of view on the situation. Unlike other systems, the format dialog in Linux will always be a little more complex because of all the options that are available, in addition the Linux user not only has to think of something that works on their platform, but also works on other platforms like Windows.
Grouping the options into „cross-platform file systems“ vs. „native linux file systems“ would help reduce descriptions and the user to be clear about what they require. For example: cross-platform file systems; FAT is the perfect legacy file system, exFAT for partitions larger than 32GB or file sharing greater than 4GB (FAT Limit) and NTFS to journaled file system compatible with Windows. For native linux file system, Ext4 is a multipurpose journaled file system, XFS enterprise journaled filesystem, Btrfs a cow file system, F2FS for flash devices, etc.
Hugo says:
13th June 2017 at 22:53
Ext4 is the most standard file system on Linux, but it is not multipurpose. It is not useful to be used in a pendrive for example. Anyway the standard in that area is microsoft lightweight file systems.
Camila Gonzáles says:
13th June 2017 at 06:21
Allan’s mockup is very good, however would make some changes. For example NTFS would put it in „others“ and instead put exFAT maybe using the same analogy as Ext4 and XFS, „volume larger than …“ compare to FAT. On the other hand, it should be considered that the format dialog will be used mostly for external devices. Describing that a file system is for internal use is rare, since others such as Btrfs or XFS can accomplish the same task. The option to create a Swap partition should be removed, as this should be the installer’s task, also major distros like Ubuntu have switched to using a Swap file.
The file systems shown in the last mockup have a clear purpose, I don’t understand why it would be necessary to display all those file system available in Udisks.
Hugo says:
13th June 2017 at 23:28
Nice work!, I thought gnome-disks was dead and it’s good that someone is interested in renewing it a little. I think that simple is better, the user should not think much about it, however, with a brief description is not enough and the acronyms can not mean anything, so it should be complemented with good piece of documentation.

GNOME Disks spins up for GSoC

Wed, 17 May 2017 12:09:55 +0000

This year’s Google Summer of Code features many valuable open source organizations and GNOME is one of them. In GNOME’s project list you can find Disks, the GNOME utility for dealing with storage devices. I’m happy that it worked out, because at first it was not in the list of ideas. The fact that Disks was considered unmaintained bothered me, but also was a big hint to finally get involved instead of mere bug reporting. So the plan emerged to propose working on two key features: Resize and repair filesystems.

Getting in contact with people through open bugs, IRC and the mailing lists went well and I’m grateful for the support (extra thanks to Michael!). Ondrej stepped in as mentor and made this possible, I already learned a lot and just took over maintainership. In the rest of this post I write a bit about the GSoC project outline and recent development in GNOME Disks.

Goal: GNOME Disks should support repairing and resizing filesystems through UDisks, thus enabling non-root users to safely set up their devices from a graphical interface.

Motivation: GParted does not use UDisks, runs only as root and thus not under Wayland.

Outline

Work out new D-Bus interface functions in cooperation with the UDisks team so that UDisks offers resize/repair through libblockdev (FS plugin) and is also tooling aware by probing relevant binaries. The exact solution is to be negotiated, since Cockpit might make use of it too.
Use that new UDisks functionality in GNOME Disks, design and implement UI (follow HIG, good progress indication)
Focus on supporting just a few filesystems in the beginning

The first task is to rework the format dialog, which should stick with the principle to offer just a few sane defaults but also not forbid to choose others (list available filesystems and find a good solution for missing tooling).

(Needs Radio Buttons for the selection.)

Development News

Translation work is constantly going on – great to see that!

I’ve set up a GNOME Wiki page as website for Disks.

A bug preventing unmounting and locking is fixed as well as a bug in the format dialog.

Taking care of loop devices:

Explain the auto-clear option, i.e. detaching loop devices if they are not mounted anymore
Disable auto-clear during operations if it would lead to unwanted detachment
Create a new empty disk image from the AppMenu

UI improvements:

Show UUID
Hints for passphrase missmatch
Indicate maximal FS label length
Explain the meaning of the mounting and encryption options, i.e. overwriting the session defaults with fstab/crypttab entries
Use mimetypes for disk image selection (welcome Mitchell as new contributor!)

There are many ways to contribute to GNOME, please support the Outreachy by spreading the word!

Design of a Python-subset Compiler in Rust targeting ZPAQL

Tue, 18 Oct 2016 22:49:42 +0000

B.Sc. Thesis, Computer Science Freie Universität Berlin, Kai Lüke

Abstract:

The compressed data container format ZPAQ embeds decompression algorithms as ZPAQL bytecode in the archive. This work contributes a Python-subset compiler written in Rust for the assembly language ZPAQL, discusses design decisions and improvements. On the way it explains ZPAQ and some theoretical and practical properties of context mixing compression by the example of compressing digits of π. As use cases for the compiler it shows a lossless compression algorithm for PNM image data, a LZ77 variant ported to Python from ZPAQL to measure compiler overhead and as most complex case an implementation of the Brotli algorithm. It aims to make the development of algorithms for ZPAQ more accessible and leverage the discussion whether the current specification limits the suitability of ZPAQ as an universal standard for compressed archives.

Download paper: Design of a Python-subset Compiler in Rust targeting ZPAQL, Kai Lüke

Converted to HTML with pdf2htmlEX: Design of a Python-subset Compiler in Rust targeting ZPAQL, Kai Lüke

Web Connectivity in Ghana - A survey

Sat, 11 Jul 2015 17:14:49 +0000

Abstract:

Starting from the research “Dissecting web latency in Ghana” of Zaki et al. the field of Internet connectivity in Ghana is explored and the different aspects like latency, DNS resolution times, complexity of web sites, caching and prefetching, TLS/SSL, TCP and cellular network performance, role of ISPs, Internet Exchange Points, are brought in relation. Measurement approaches and tailored solutions for areas with bad connectivity are listed. Possible ways of improvement are discussed.

Download paper: Web Connectivity in Ghana - A survey, Kai Lüke

Converted to HTML with pdf2htmlEX: Web Connectivity in Ghana - A survey, Kai Lüke

Categories and Subject Descriptors: C.2.6 [Internetworking]: Standards; C.4 [Performance of Systems]: Reliability, availability, and serviceability

Keywords: Developing Countries, Internet, Latency, Performance, TCP, Mobile, Cellular

modifying kernel space - uname hack revamped

Tue, 27 Aug 2013 01:00:12 +0000

This change of the original uname hack kernel module was laying unreleased and forgotten on the disk, but after some tweaking to support all options, here it comes — ugly and not meant for serious usage:

uname_kernel_hack_module on github

Some months ago I modified it to not change the machine from e.g. amd64 to i386 but the kernel version. Curiously to solve an issue it itself is having in the second line of the Makefile (uname -r); some actions rely on the kernel version to access the actual initramdisk or kernel headers. These might be different versions within guest system containers and led to the ugly hack of overwriting the kernel struct (yes, we are on Linux, not GNU Hurd) where the uname tool gets it’s release version from:

struct new_utsname {
char sysname[__NEW_UTS_LEN + 1];
char nodename[__NEW_UTS_LEN + 1];
char release[__NEW_UTS_LEN + 1];
char version[__NEW_UTS_LEN + 1];
char machine[__NEW_UTS_LEN + 1];
char domainname[__NEW_UTS_LEN + 1];
};

At initialisation the module saves the original value of e.g. sysname and overwrites the kernel space struct with the value you want. After usage, when the module is unloaded, the original value will be restored.

So when using the provided Makefile you could test how your system looks like when pretending to be something else:

make KRELEASE=2.5.2-0 KSYSNAME=Minix KVERSION="Debian 2.5 (2012-03-04)" KMACHINE=i386 test

The output while running make could look like:

uname -a
Linux hstnm 3.10-2-amd64 #1 SMP Debian 3.10.5-1 (2013-08-07) x86_64 GNU/Linux
sudo /sbin/insmod unamehack.ko
[sudo] password for kai: 
uname -a
Minix hstnm 2.5.2-0 Debian 2.5 (2012-03-04) i386 GNU/Linux
sudo /sbin/rmmod unamehack
uname -a
Linux hstnm 3.10-2-amd64 #1 SMP Debian 3.10.5-1 (2013-08-07) x86_64 GNU/Linux

So a longer usage would require you to enter

make KMACHINE=i386 insertunamemodule

and revert the changes as soon as possible by

make removeunamemodule

So, good luck! You’ll better not use it and try to fix cases where it’s necessary.

„Faire“ Computer?

Sun, 10 Jun 2012 15:22:28 +0000

Foxconn bekommt durch Apple in letzter Zeit immer mehr Aufmerksamkeit bezüglich der Gewerkschaftsrechte und Arbeitsbedingungen der Fabriken in China. Gerade weil iLifestyle und Suizid so weit voneinander weg zu liegen scheinen. Kampagnen wie makeITfair (Germanwatch e.V.) u.a. sowie der Greenpeace »Guide to Greener Electronics« werfen den Blick auf die Schattenseiten der Technik in Herkunft der Materialien, Produktionsbedingungen, Giftstoffen, Energieverbrauch, Entsorgung und die mitgekaufte Verantwortung. Obwohl auch in Deutschland Reuse vor Recycle gilt, wird meist schon das Recycling in den Anfängen der abwechselnd von den Herstellern organisierten Entsorgungskette erstickt.

Gestern sah ich den Film »Blood in the Mobile«, der die für Elektronikprodukte essentiellen »Konfliktmineralien« aus dem Kongo zum Thema hat. Aufgebracht durch einen UN-Expertenbericht, der aussagt, dass ein direkter Zusammenhang zwischen Minenabbau für Mobiltelefone und Angriffen auf die Bevölkerung durch Milizen besteht (UN Group of Experts, Dez. 2008, S/2008/43), tritt Regisseur Frank Piasecki Poulsen mit dem Mobiltelefonhersteller seiner Wahl in Kontakt. Aber woher die verwendeten Mineralien stammen, kann und will niemand sagen, obwohl seit 10 Jahren das Problem intern diskutiert wird - zudem sei auch die ganze Elektroniksparte betroffen und auch bei den Konsumierenden läge die Verantwortung. Poulsen will verstehen und mit eigenen Augen sehen, wie die Realität vor Ort aussieht und bereist mit Kamerateam eine der Minen und zeigt somit die verschiedenen Problematiken auf. Lösungsansätze werden auch diskutiert und als Anfang ist durch einen US-Gesetzesantrag erreicht worden, dass die Zulieferungskette transparent veröffentlicht werden muss. Eine Projektseite ist hier.

Relativ neu ist die Doku »Behind the Screen: Das Leben meines Computers«, welche einige Aspekte der Herstellung und Entsorgung beispielhaft abdeckt. Da EU-Exporte für Elektroschrott verboten sind, landen als Gebrauchtware deklarierte Müllladungen auf Halden in Ghana und werden in Accra, Koforidua usw. zerlegt. Dieser Prozess setzt höchstgiftige Stoffe frei und betrifft somit die dort arbeitenden Kinder mehrfach. Auch die Bleikonzentration und weitere Kunststoffe wie Weichmacher belasten das Trinkwasser.

Doch bevor ein Device dort landet, wird es z.B. in Tschechien von vietnamesischen Leiharbeitern in Knebelverträgen hergestellt. Das verwendete Gold könnte aber wieder aus Ghana kommen. Hauptsache, wir sind auf der „richtigen“ Seite des digital divide? Mehr zum Thema Informatik & Gesellschaft auch hier, in der Zeitschrift des »Forums InformatikerInnen für Frieden und gesellschaftliche Verantwortung e.V.«.

Codecvergleich mit GStreamer durch SSIM

Fri, 18 May 2012 01:09:31 +0000

Während der Wirrungen bezüglich des HTML5-Videostandards hatte ich (noch lange bevor WebM existierte) einen Blick auf Dirac geworfen, einen alternativen Codec der BBC Research Labs. Nach subjektiven Vergleichen und einiger Benutzung, fehlte mir dennoch ein Verfahren, um mehr verlässliche Aussagen über die Qualität (sowie zu erwartende Größe) eines komprimierten Videos zu machen. Anstelle des „alten“ PSNR wurde SSIM zur Differenzanalyse zwischen Original und dem komprimierten Bild verwendet. GStreamer bietet sich durch starke Flexibilität an - ein Testframework wurde in Python mit Django umgesetzt. Leider steht der Out-of-the-box-Verwendung noch einiges im Wege; gerade die SSIM-Berechnung ist noch nicht als eigenständiges GStreamer-Element implementiert, was Geschwindigkeit und Stabilität verbessern würden. Auch die generelle Anbindung der GStreamer-Mainloop in die Webapplikation wäre besser durch einen daemon zu lösen. Daher kommt hier jetzt noch kein Quellcode…

Ausarbeitung (Kurs Digitales Video): Der Dirac-Wavelet-Codec und automatisierter Qualitätsvergleich mit Hilfe des GStreamer-Frameworks

Die erhobenen Daten sind nur beispielhaft und erheben keinen Anspruch auf vollständige Charakterisierung der Codecs. Denn der Zeitaufwand für SSIM ist groß - und somit ist die Eingabesequenz sehr kurz gewählt.

Arduino N64 Controller Library und Tetris-Port

Mon, 26 Mar 2012 02:24:05 +0000

Moved to GitHub: https://github.com/pothos/arduino-n64-controller-library, except for N64Tetris

Die Arduino-Plattform (arduino.cc) bietet einen einfachen Einstieg in die Microcontroller-Programmierung und ist gerade im Nicht-Geek-Bereich eine Erfolgsgeschichte. Die vielen Bibliotheken ermöglichen es, hauptsächlich auf der “Organisationsebene” zu bleiben - in kurzer Zeit war aus der Kombination von TVout, dem verfügbaren Tetris-Clone und einigen Modifikationen zum Speichern der Highscore im EEPROM eine archaische Spielkonsole aufgebaut. Dank dieser Anleitung konnte ein N64-Controller benutzt werden. Während für NES relativ viel Quellcode verfügbar ist, gibt es keine komfortable N64-Controller-Library, da wegen exaktem Timing Inline-Assembler verwendet wird und der PIN fest eingeschrieben ist. Meine Bibliothek ist vielleicht nicht in schönster Manier geschrieben und alles andere als optimal in der Codegröße, aber für eine Erstveröffentlichung sollte es ausreichen - komfortabel ist sie zumindest (auch alle möglichen PINs von 0 bis 13 können unabhängig voneinander für Controller benutzt werden).

Die Lizenz ist unklar, es handelt sich hauptsächlich um Modifikationen und Zusammenstellungen. Der Tetris-Port “Simple Tetris Clone” ist z.B. unter der MIT-License veröffentlicht, die meisten Bibliotheken unter der (L)GPL.

Version ~~1 der Library (26.03.2012)~~ 2 der Library (26.07.2012),

getestet auf Arduino Uno: N64Controller

Hier der Quelltext für das Spiel: N64Tetris

Beispielquellcode:

#include <N64Controller.h>

N64Controller player1 (12); // PIN 12

void setup() {
    player1.begin(); // Initialisierung
}

void loop() {
    delay(30);
    player1.update(); // Tastenzustand auslesen
    if (player1.button_A() && player1.button_D_down()
        || player1.button_Start()) { // gelesene Tasten abfragen

        int xachse = player1.axis_x(); // kann negativ oder positiv sein,
                                       // je nach Ausrichtung des Analog-Sticks
    }

    // …
}

Facharbeit

Sat, 23 Oct 2010 18:49:19 +0000

Ich habe meine Facharbeit in Mathe geschrieben zum Thema Public-Key-Kryptographie am Beispiel des RSA-Verfahrens: heutige Umsetzung und Implementierung.

Hier sind die Links:

Facharbeit als PDF

schnelles hybrides Verschlüsselungs-Programm aufbauend auf dem Facharbeits-Code (src)

das kompilierte Programm für Linux (64 Bit) braucht die Ubuntu-Pakete libgmp3c2 libgmpxx4ldbl und zum Kompilieren libgmp3-dev

und das kompilierte Programm für Linux (32 Bit)

das kompilierte Programm für Windows