Recent work

A lot of work has recently been going on in terms of Secure Boot support on Fedora. Recently we’ve set up builders with a special configuration for signing appropriate packages, provisioned keys onto hardware cryptographic devices to keep them safe in the event of an intrusion, and started building packages with our production signing keys.

What’s next

The next step will be to have shim signed by the UEFI signing service. But there’s still much that can be done before we cross that threshold, and we could use help from those that have appropriate hardware to test with. Today, Fedora-signed packages for shim and grub2 have been built, and they’ve got updates filed in Bodhi for Fedora 18: here for shim and here for grub2. Sometime soon - most likely tomorrow - we’ll build a kernel that will be signed by the production keys as well. You can tell if it’s a signed kernel by viewing the build log. Look for this:

+ ‘[’ -x /usr/bin/pesign -a x86_64 == x86_64 ‘]’
+ ‘[’ -e /var/run/pesign/socket ‘]’
+ /usr/bin/pesign-client -t ‘OpenSC Card (Fedora Signer)’ -c
‘/CN=Fedora Secure Boot Signer’ -i gcdx64.efi.orig -o gcdx64.efi -s

If the “-t” argument to pesign-client says “OpenSC Card (Fedora Signer)”, then it’s the correct build. I’ll post an update here with the relevant kernel build as well.

Once that happens, those of you who already have Fedora 18 test composes installed using UEFI on hardware that supports secure boot can help test.

The test procedure

First, ensure that your machine is in “setup” mode - that is, Secure Boot is disabled, and no keys are enrolled. Once you’ve done that, install the updated shim, grub2, and kernel packages, and then it’s time to install some new keys. Download this utility into /boot/efi/EFI/fedora/ . Now you’ll need to run this utility from your UEFI firmware. Unfortunately how to do that will vary across many systems, so you’re on your own figuring that out. Once you’ve run it, reboot the machine one more time so that the settings are live.

Now the machine should be booting in Secure Boot mode signed with the Fedora keys.

One thing we’ve heard several times during the discussions of the UEFI “Secure Boot” mechanism is that there’s no genuine security benefits. In fact there are several. The ability to guard against “bootkit” systems has been covered elsewhere, so I’ll not cover it here. Instead, I’d like to focus on a less publicized aspect of Secure Boot’s mechanism.

With secure boot, it’s possible to verify any UEFI application which has been signed. There’s a reasonable standard of attestation that if a binary has been signed by our key, it was consciously signed by us.

A simple example

One side effect of this is that with a few hoops, we can sign arbitrary data, and wrap it in a UEFI envelope. Such code would look something like:

1
2
3
4
5
6
7
8
9
10
11
12

#include <efi.h>
#include <efilib.h>

EFI_STATUS
efi_main (EFI_HANDLE image_handle, EFI_SYSTEM_TABLE *systab)
{
  InitializeLib(image_handle, systab);
  Print(L"This program contains data.\n");
  return EFI_SUCCESS;
}

int data __attribute__ ((__section__(".data.hellothere"))) = 3;

At this point you’d build it like any gnu-efi application, and then have it signed as a UEFI application.

At this point you’ve got an envelope for data which is signed and explicitly trusted by your hardware platform. At runtime, you can verify the signature against DB, the UEFI key database, and verify that this is a trusted binary. At that point in time you can unwrap the UEFI binary to get at the ELF DSO that fundamentally lies underneath, and examine the DSO. If you do “eu-readelf –section-headers ${file}” on the DSO, you’ll see that there are 21 section headers. Here’s a subset:

Section Headers:
[Nr] Name Type Addr Off Size ES Flags Lk Inf Al
[ 0] NULL 0000000000000000 00000000 00000000 0 0 0 0
[ 1] .hash HASH 0000000000000000 00200000 00000690 4 A 9 0 8
[ 2] .eh_frame PROGBITS 0000000000001000 00201000 000016b0 0 A 0 0 8
[ 3] .text PROGBITS 0000000000003000 00203000 000053bd 0 AX 0 0 16
[ 4] .reloc PROGBITS 0000000000009000 00209000 0000000a 0 A 0 0 1
[ 5] .data.hellothere PROGBITS 000000000000a000 0020a000 00000004 0 WA 0 0 4
[ 6] .data PROGBITS 000000000000a020 0020a020 00001770 8 WA 0 0 32

.data.hellothere starts at 0x20a000 and is 4 bytes long.

Something with more substance

What’s this useful for, you ask? Well, let’s assume we’ve got a slightly different binary:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

#include <efi.h>
#include <efilib.h>

EFI_STATUS
efi_main (EFI_HANDLE image_handle, EFI_SYSTEM_TABLE *systab)
{
  InitializeLib(image_handle, systab);
  Print(L"This program contains a list of public keys.\n");
  return EFI_SUCCESS;
}

struct key {
  int size;
};

struct keylist {
  int numkeys;
};

#define KEYDATA \
"-----BEGIN PGP PUBLIC KEY BLOCK-----\n" \
"Version: GnuPG v1.4.5 (GNU/Linux)\n" \
"\n" \
"mQINBE8MkeIBEADfn8QtElquxUAVvS8th7UnVmfkExw7jC7SVy7dGlXo5rgxqETM\n" \
"AZorpDIvAtKX5VMr/lUfODB6sSymh0e6EdvYQfHrpImO05F7WBq2DhRW74j1DRNu\n" \
"8vohKPTsSZTEZo/mrUBDAAGtGOXcrsQxx4J2Ur73+r18ODd+v6O33YevlFmwmFYT\n" \
"uDcgdhxyxBdpJVES8MdxO349uP9bvrU+3KjpDDGHb/hMbY8az7lwLtBufRrAekhN\n" \
"5Cg7+zm+I+wVGzgzSw0yrIh4hdVts4RKZIrl2N3VeZcCY4IrNZFd2Do6HhIwA7l/\n" \
"/xpTEBTZ4BmnGP9iufUVa2h97/JmKy38rM88IovIcTSnYDG68k/NxUkC+dLriI4T\n" \
"BNc9kJLVo821x77WYViNXHscF8ujlf73HilfCnhEtNwViGO7x41guQrdv5k79UDf\n" \
"1CylSZ+76vKXziz6uNHzIagiViNOvYHOoEH8jDNnubqFTxYXUNks5l3/byllQ0XK\n" \
"Oj9RarmjsMS3udRB6RCiEnzDbPge2S9gmdmY8MA3QeZ3aWoZJqygDQ0WNUOjPZ0E\n" \
"80x+xbUyAK06/SCzs6bUx6GMXd5Iy5r3Gc+6LRt4WEG5r/vJEHPSj3qAoVqydsY0\n" \
"GjASPIQmYduCdi7Inf0w+2n+nJGPcK1k432xHD0z0kp1Q/xcOH/Tul3bcQARAQAB\n" \
"tCZGZWRvcmEgKDE3KSA8ZmVkb3JhQGZlZG9yYXByb2plY3Qub3JnPokCNgQTAQIA\n" \
"IAUCTwyR4gIbDwYLCQgHAwIEFQIIAwQWAgMBAh4BAheAAAoJEFDpTJkayjRlKxAQ\n" \
"AICtQT+j8Sum1J86yBwso1wuzK2sHOXbfC1LCdQw2u6QLfKSUsCnQ5AL+oCnS491\n" \
"5fjrYksT2siclLhgZX7/yF76XuYHHhRTO65NaPSCxxhN6S9zbExUPRoxFL1ay0cH\n" \
"p14WYI1/SGmKPcJmigV6n8+wGYl+zWlH9eiiFP/+JCxJ0ZvRg3mgT5zcPdIzTDOz\n" \
"4rTE3WfH3qqxYhw2ttDPomBSdbgJw2N6bj46t8rljIDGdeKpFHYMVXpUp0gDkEoY\n" \
"bAiQhvZaa2F6mGMfXRdr4Phs562+tF/Qy/JaK8uafYZPHTtC16NHf87DYceLzvci\n" \
"W1KNBYGrEHm5NMguBlqP348FYMp+6hJDYMl38Qx3BK9bz6lW0G75tgaAZ+sxAFGs\n" \
"/MUcjChkF9OcF7Y7W6hY3IQb/+FgB5eNpBqfmZZ76ywpS+D0sIqxzWoILtlXdHeN\n" \
"eM4YiX0gvVQXwn9S89O5vWKxWYspywfgV/aXHk14O6k7oi8wInJOvgxZKEHUT4XU\n" \
"K7DyogL45VV9iYoWYIym2L17VBnINE/Kwmc/81nYigE0tTOwaR+qMFBSRlkOA4+X\n" \
"ZlIvb5Cft7CdC183FYLIM+B6xNSKE/OhavHZsRJrLZC1aAP8MNh8Cy7Jqrn1/Qjj\n" \
"6sncrqaj/Yis4yAaINk4MrnbImN2MwzU70q0eR3kA6PY\n" \
"=3+4G\n" \
"-----END PGP PUBLIC KEY BLOCK-----\n"

struct keylist keyhdr __attribute__ ((__section__(".keylist"))) =
  { 1 };
struct key key __attribute__ ((__section__(".keylist"))) =
  { sizeof(KEYDATA) };
uint8_t pubkey[] __attribute__ ((__section__(".keylist"))) = KEYDATA ;

With this code, the extracted DSO has a section like this:

[ 5] .keylist PROGBITS 000000000000a000 0020a000 00000680 0 WA 0 0 32

But wait, there’s more

One additional component here is packaging. If we package the file on a per-repo basis, with a well known name, there’s a bit more.

So let’s say, for example, that we can have ${repo}-release provide a file, /etc/pki/sbkeys/${repo}-keys.efi, which looks roughly like what’s shown above.

At this point, we can add the extraction code to anaconda and kickstart, and we can automatically import trusted keys for any repository that includes a correctly signed keyfile. This is bug 253897, a bug that’s been open for 5 years.

Wait for it

There’s an even larger goal here - bug 253897 is the remaining blocker on bug 998, open since 1999.

With the above system, we can enable signature checking for all packages during installation.

One of the many things I work on is UEFI support. It’s an interesting thing to work on, in part because there’s a lot of new development and it’s at a fairly low level, which is just the sort of thing I like.

Often during UEFI development, we’ll see a bug and need to diagnose whether it’s a problem with the hardware, the firmware, the bootloader, the OS kernel, or even a userland program. One case of this is when console graphics don’t work right.

A recent bug reported that console graphics weren’t working due to an invalid pixel format being chosen. While diagnosing this problem, there are a couple of things that could go wrong. The firmware driver could be reporting the wrong modes, the bootloader could have a bug where it’s querying the modes incorrectly or misinforming the kernel as to the mode chosen, or the kernel could have a bug where it uses the mode incorrectly. It’s also possible that the console works until a KMS driver - that is, a native driver for the card - is loaded.

In UEFI, console graphics involves every part of the stack. The system firmware provides an API known as Graphics Output Protocol, or GOP. The algorithm used by the bootloader basically looks like this somewhat simplified function:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

int gop_enable(struct graphics *graphics, struct kernel_params *kernel_params) {
  efi_handle gop_intf = efi_locate_protocol(&graphics_output_guid, NULL);
  efi_status status = EFI_SUCCESS;

  if (!gop_intf)
      return -1;

  graphics->max_mode = gop_intf->mode->max_mode;
  graphics->modes = calloc(graphics->max_mode, sizeof(void *));
  if (!graphics->modes)
      return -1;

  for (int i = 0; i < graphics->max_mode; i++) {
      graphics->modes[i] = calloc(1, sizeof graphics->modes[0]);
      if (!graphics->modes[i]) {
          status = EFI_NO_MEMORY;
          goto fail;
      }

      graphics->modes[i]->number = i;
      status = gop_intf->query_mode(gop_intf, i,
                      &graphics->modes[i]->size,
                      &graphics->modes[i]->info);
      if (status != EFI_SUCCESS) {
          free(graphics->modes[i]);
          graphics->modes[i] = NULL;
          goto fail;
      }
  }

  /* this sorts the modes by size, but leaves mode.number intact
  * so that we have a way to reference it to the firmware */
  sort_modes(graphics->modes);

  gop_intf->set_mode(gop_intf, graphics->modes[0]->number);
  set_kernel_params(graphics->modes[0], kernel_params);

  return 0;
fail:
  /* best effort... */
  if (graphics->modes && graphics->modes[0]) {
      gop_intf->set_mode(gop_intf, graphics->modes[0]->number);
      set_kernel_params(graphics->modes[0], kernel_params);
  }
  return -1;
}

This is a simplified version - for more details, check out efigraph.c for the excrutiating details.

set_kernel_params() in turn fills out the data structure we give to the kenrel, which then uses the efifb driver to configure a framebuffer console with the correct resolution and pixel format.

Clearly, there is plenty that can go wrong here. With that in mind, I wrote a tool called modelist.efi to debug some of these problems.

In the bug, I’m told:

With a 1280x1024 capable monitor connected to a *censored* or *censored* in
uEFI mode, the remote video redirection via BMC (*censored*) does not display
anything. The reason is that the efifb driver chooses the VGA mode
1280x1024x32bpp which the video redirection doesn’t support (the *censored*
chip doesn’t support video modes that need more than 4MB video RAM).
See also bug #XXXXXX for the same problem reported in the past for Xorg / mga
driver.

efifb: probing for efifb
efifb: framebuffer at 0x90000000, mapped to 0xffffc90011800000, using 8192k,
total 8192k
efifb: mode is 1280x1024x32, linelength=5120, pages=1
efifb: scrolling: redraw
efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
fbcon: EFI VGA (fb0) is primary device
Console: switching to colour frame buffer device 160x64
fb0: EFI VGA frame buffer device

With the knowledge of how the stack works in mind, I know that we need to find out what the firmware is actually returning to the bootloader in order to tell what layer the problem is at, so I ask:

Can you run /boot/efi/EFI/redhat/modelist.efi (from the gnu-efi package) and
show us the output?

And the response is:

fs0:\EFI\redhat> modelist.efi
GOP reports MaxMode 4
0: 640x480 BGRR pitch 640
1: 800x600 BGRR pitch 800
2: 1024x768 BGRR pitch 1024
3: 1280x1024 BGRR pitch 1280

So this tells me that the firmware is listing 4 video modes, each of which has a 32-bit pixel of the form (blue,green,red,reserved). At this point we know that the firmware is telling us about modes that the hardware cannot support, and efifb cannot correctly support this device until the firmware is fixed.

It’s important to have and use the right tools.

The Economist has run an article this week that’s left me quite disappointed. In A tangled web, they make an attempt to explain the question of Net Neutrality without taking sides. In doing so, some critical details of the history of the debate are omitted and biased terms are allowed to stand without serious discussion. This leaves the reader in a position where forming an unbiased understanding of the facts is impossible. The result is a rather more biased viewpoint than I’ve come to expect from The Economist. The article’s major failings are twofold. It forms a broad analogy which doesn’t hold up to scrutiny - that “fast-tracking” and “pay for priority” are the same thing - and fails to appropriately distinguish between backbone “transit” (bandwidth between ISPs) and end-user bandwidth.

Fast tracking, they explain, is the practice of being able to pay more money to an ISP to get a faster internet connection. “Pay for priority”, in contrast, is a term used by service providers to describe selling ad-hoc services to a consumer. The problem with the latter is in the details. To implement “pay for priority”, they must limit bandwidth on a per-service basis for other services - either at the transit level or at routers that service individual consumer connections. This necessarily means interfering with network connections, stifling connections that don’t originate at their services in order to provide services to their customers faster. There are several methods which can be utilized for this, ranging from simply providing insufficient transit bandwidth to actively tampering with user’s connections.

The transit method requires that the ISP is also fulfilling the role as content provider, which is the normal case in practice. This method is to simply have much more bandwidth between the ISP’s content servers and the customer than the amount of bandwidth customers demand to other content providers, or to artificially restrict bandwidth between the ISP customers and outside content. This is what they refer to as “reasonable network management”, but it’s anything but reasonable. Under this scenario, an ISP customer buys (for example) 20Mb/s of download bandwidth. But what the customer actually receives is 20Mb/s of download bandwidth from content their ISP hosts, and severely limited bandwidth to other content providers. This is essentially what Comcast wants to do with Netflix bandwidth - they want Netflix’s ISP, Level 3, to pay extra to send data to Comcast’s internet customers, or Comcast will limit their customers’ access to Netflix’s streaming video data.

What the cable companies want agency to do concerning their ISP customers, however, is not limited to merely extortive practices with other content providers’ data. Nor have they attempted to charge their ISP customers more in order to provide more bandwidth to outside content. When Comcast decided that bittorrent was in competition with their TV and movie content interests, for example, they didn’t simply block all bittorrent traffic and demand a fee from their ISP customers for access to the service. In this case there was no individual company from which to extract fees, since bittorrent is a so-called peer-to-peer service, where traffic originates at individual ISP customers both inside and outside of Comcast’s network. Instead, they intentionally modified their ISP customers’ bittorrent traffic by introducing fake packets which reset their customers’ connections, dramatically increasing the protocol overhead for any bittorrent users to the point where the service was not usable. The customer saw what was a technically functioning service, but with performance so degraded that nobody would want to use it. This is what “pay for priority” means in practice. In short, it’s institutionalized fraud: in theory the consumer is paying for bandwidth, a commodity, from an ISP, but in practice the ISP has control over what the customer can use that bandwidth for - thus relieving it of commodity status.

While the article’s conclusion - that lack of competition among service providers is a large problem which must be addressed - is correct, the authors fail to understand that allowing service providers to discriminate among services being carried through the connections leaves an Internet where price discrimination for access to bandwidth is irrelevant.