scaramanga

Annotated Version Of Liz Truss Resignation Speech

2022-10-22T07:38:36+00:00

Some say she’s the worst prime minister in UK history. UK history isn’t very long so they probably mean English history. Either way, I think that would be a hasty judgement. It could easily be the case that she has set in motion the destruction of the conservative party as a viable force in English politics. That would make her probably the best prime minister in history. The competititon isn’t exactly stiff. What follows is an annotated version of the resignation speech given by Liz Truss. I hadn’t seen any useful commentary of this anywhere so I thought I’d add my own. The speech was impressive, not so much in it’s content, but more for her ability to forcibly grin like an imbecile for that long.

I came into office at a time of great economic and international instability.

Your party has been in power for 12 years.

Families and businesses were worried about how to pay their bills,

Your party has been in power for 12 years.

Putin’s illegal war in Ukraine threatens the security of our whole continent

Irrelevant. But no, it doesn’t.

and our country has been held back for too long by low economic growth.

Thanks to your party, which has been in power for 12 years.

I was elected by the Conservative Party with a mandate to change this.

Just not a democratic mandate.

We delivered on energy bills and on cutting National Insurance.

No, you did not. You proposed to pay people’s energy bill by borrowing money that they would have to pay back with interest.

And we set out a vision for a low tax, high growth economy that would take advantage of the freedoms of Brexit.

You say “vision”. The world says “delusion”.

I recognise though, given the situation, I cannot deliver the mandate on which I was elected by the Conservative Party.

Nobody cares what they want. They can all die in a fire, and it would be their own fault for not leaving the fire.

I have therefore spoken to His Majesty the King to notify him that I am resigning as leader of the Conservative Party.

A touching moment, shared between two unelected leaders.

This morning I met the chairman of the 1922 Committee, Sir Graham Brady.

A body with less constitutional validity than the Iranian guardian council.

We’ve agreed that there will be a leadership election, to be completed within the next week.

An irrelevant party-political trifle.

This will ensure that we remain on a path to deliver our fiscal plans and maintain our country’s economic stability and national security.

By this, I presume you mean “maintaining the current rate of rapid decline”.

I will remain as prime minister until a successor has been chosen.

That you yet cling to this amount of power (like a fucking tick) is, in itself, a disease of our political system (limes disease, I suppose).

Thank you.

For what?!

Anyone who has ever installed fedora virtio-win packages via yum is vulnerable

2020-12-23T03:40:00+00:00

If you have ever installed the virtio-win package in accordance with the instructions from fedora, then your system is vulnerable to a remote root exploit every time you do a yum upgrade, and will stay like that forever more, until you remove the virtio-win yum repo.

That is because the instructions tell you to setup a yum repo with the with a plain http baseurl and gpgcheck=0… This, essentially, disables all security and allows a network man-in-the-middle (even a regular old web proxy) to inject bogus updates every time you do a dnf upgrade - which will be nightly if you have installed dnf-automatic.

This entire process, including exploitation, can happen without dnf providing any hint to the user that it is downloading arbitrary code insecurely over the internet and running it.

What can you do about it?

Turns out that there is not a whole lot you can do about this yet. My best advice to you is to disable this yum repo immediately.

I reported this bug in September but it turns out it had already been reported in January 2016.

There is a github issue tracking the change required to sign the contents of virtio-win repository.

Windows has a better security culture than Linux

Note, that the windows drivers themselves are signed. That’s because loading windows drivers without signatures is a complete rigmarole so the drivers would be next to useless if they weren’t signed.

But here, the signatures are luring people in to a false sense of security because you may imagine that since all RPM is doing is downloading some opaque blobs which only a guest will install, that as long as the blobs are signed and the windows guest OS verifies them then an end-to-end root of trust is established.

But this is not so, since the Linux host is compromised. All an attacker has to do is create a new RPM based on the old virtio-win RPMs (including all the valid signatures) and just add a %postinstall RPM script which roots the host. So now it doesn’t matter that the guest is secure, because the host is compromised. And if the host is compromised then all bets about the integrity of the guest are off.

If RPM applied its signatures with the same dilligence as Windows update, we probably would not be in this mess.

I also filed a bug against dnf that it insecurely installs code from the internet without so much as a warning message.

DNF does not check TLS certs if you use https! (not true)

Initially I thought that a quick workaround would be to change the repo URL to https. After all, fedoraproject.org should be trustworthy.

But according to the DNF developers “DNF is happy with any HTTPS connection, even with a self-signed certificate and a complete redirect to a different server would be unnoticed.”

Apparently there is a plan to solve this but it won’t land before RHEL9 because the user experience would necessarily change if dnf were to check HTTPS certificates.

Of course, there are good historical reasons not to use TLS, it breaks caching. But given todays threat landscape, mirror services may just want to suck up this cost.

Update: provided that sslverify=1, which is the default, the yum TLS client will reject any connection that cannot be authenticated and downloads will fail.

Arbitrary remote code execution is arbitrary remote code execution

Merry christmas, happy new year, it’ll soon be 2021, and here’s to another year of telling people that arbitrary remote code execution is arbitrary remote code execution.

You can refer to the virtio-win repo issue as CVE-2020-29665.

Edits

Originally the title was sloppily written and it implied that all virtio-win packages available on any yum repo had this problem. In fact, the fedora repo is only once such repo. Apparently the RHEL repos do not suffer from this problem. Thanks Cole Robinson for pointing that out.

Thanks to Ken Dreyer and Daniel Mach for clearing up the points about DNF.

A Faster Partition Function in Python

2020-12-14T14:22:00+00:00

Some pythonistas wonder what is the fastest way to write a function to partition a sequence of items in to two lists based on some predicate. One could, to be sure, filter the list and then filterfalse the list. But in this case the predicate will evaluated twice for each item - an inefficiency that some would balk at. That approach also doesn’t seem very parsimonious.

This problem has inspired several broad classes of solutions based on different needs and priorities and some very minor controversy about which is the most efficient. I hope to clear up the question of efficiency by introducing a new variant here today.

An Itch You Just Can’t Scratch

Have you ever found yourself doing something like this:

foos = [x for x in input if x.type in known_foos]
bars = [x for x in input if x.type not in known_foos]

It’s a DRY itch. Quite irritating. It calls for a salve. Surely Python would have a solution for this? It has an expansive standard library…

Could I Even Give Two Forks?

The suggestion in the itertools documentation is to use tee in an elegant way, along with filter and filterfalse, which are also both in the standard libraries. This approach also has the benefit of working with arbitrary generators. It can even, in principle, handle an infinitely long input sequence.

It’s quite clever, so it’s worth mentioning.

from itertools import tee, filterfalse

def partition(pred, it): # stdlib
    a, b = tee(it)
    return filterfalse(pred, a), filter(pred, b)

But the problem is that it’s also quite slow when you’re just going to be materialising the results immediately. To see why, imagine calling it:

hay, needles = partition(lambda item: item in needle_set, haystack)
return Results(list(hay), list(needles))

As soon as we materialise one of the returned generators, tee will internally materialise haystack in to a temporary list. And the predicates are still being evaluated twice, once for each fork of the tee.

But I can see this sort of thing being useful in a graph of asyncio tasks or something. Maybe one day I will find myself in a situation where I would need to use a cool technique like this.

The Clever Trevors

But let’s forget about possibly-infinite generators and lazy evaluation and just consider the case of a list that we want to partition in to two new lists.

For that case case, I’ve seen a couple of clever approaches.

The first one is quite quick to type and nice and readable. It involves using the result of the predicate function to index in to the results tuple. It takes advantage of the fact that True and False cast to 1 and 0 when used to index a tuple.

def partition(pred, it): # trevor
    ret = ([], [])
    for item in it:
        ret[pred(item)].append(item)
    return ret

Honestly, this probably would have been my go-to approach, but it actually turns out to be quite slow.

The second is a very functional-style solution, straight out of some sort of Haskellers PhD thesis, proposed by stackoverflow member Mariy. This one uses functools.reduce, which is pythons version of foldl:

def partition(pred, it): # mariy
    return reduce(lambda x, y: x[pred(y)].append(y) or x, it, ([], []))

In princple, this is the same as the above but it takes advantage of the fact that reduce performs some of the boilerplate for you.

I expect it to be a bit less efficient since it creates as many result tuples as there are elements in the input list. Repetetive object creation and destruction is a common source of overhead in python.

The Plain Jane

According to gboffi, it turns out the fastest solution that stackoverflow came up with is just the straight forward version, credited to Mark Byers:

def partition(pred, it): # byers
    ts = []
    fs = []
    for item in it:
        if pred(item):
            ts.append(item)
        else:
            fs.append(item)
    return fs, ts

But, knowing something about optimizing python code, I can see a clear opportunity to speed this up.

Don’t dis My Code, Man

We can disassemble this to python bytecode with dis. Generally speaking, the main determinant of performance in these kind of loops is the number of bytecode instructions which need to be dispatched per iteration.

So here is what the Byers version compiles to in cpython 3.9:

  2           0 BUILD_LIST               0
              2 STORE_FAST               2 (ts)

  3           4 BUILD_LIST               0
              6 STORE_FAST               3 (fs)

  4           8 LOAD_FAST                1 (it)
             10 GET_ITER
        >>   12 FOR_ITER                34 (to 48)
             14 STORE_FAST               4 (item)

  5          16 LOAD_FAST                0 (pred)
             18 LOAD_FAST                4 (item)
             20 CALL_FUNCTION            1
             22 POP_JUMP_IF_FALSE       36

  6          24 LOAD_FAST                2 (ts)
             26 LOAD_METHOD              0 (append)
             28 LOAD_FAST                4 (item)
             30 CALL_METHOD              1
             32 POP_TOP
             34 JUMP_ABSOLUTE           12

  8     >>   36 LOAD_FAST                1 (fs)
             38 LOAD_METHOD              0 (append)
             40 LOAD_FAST                4 (item)
             42 CALL_METHOD              1
             44 POP_TOP
             46 JUMP_ABSOLUTE           12

  9     >>   48 LOAD_FAST                3 (fs)
             50 LOAD_FAST                2 (ts)
             52 BUILD_TUPLE              2
             54 RETURN_VALUE

The inner loop is between locations 12 and 46. It’s 18 instructions long.

We can see that the append code has some duplicated instructions in each branch (24-34, and 36-46), but since only one branch is taken at a time, that shouldn’t really be counted.

So if we measure the actual instruction path-length per iteration, it comes out to 12 instructions.

More importantly, we can see that inside that inner loop we’re doing a lookup of the append method, which is actually looking up a string in a dictionary. For this reason, the LOAD_GLOBAL and LOAD_METHOD calls are among the slowest of the basic instruction types in the python machine (not including the instructions which call out to python subroutines, of course).

Python is Dynamic, and There is no Escape(-Analysis)

We have probably all heard that python is a dynamic language. But we may not know what all of the consequences of that fact are. For one thing, it means that attribute lookups are often expensive dictionary lookups. But it also means that even some very obvious optimisations simply cannot be made. For example, the python compiler cannot, in general, optimise:

# Program A
x = []
for a in b:
    x.append(a)

in to:

# Program B
x = []
f = x.append
for a in b:
    f(a)

The reason for this is that the method list.append is absolutely free to assign a different value to self.append. This means that programs A and B are (or at least, may be) semantically different. Therefore B is not a valid optimization of A.

Now you might think that, well, the bytecode compiler can know about the implementation of list.append because it’s built-in. Which is true. But it doesn’t stop any other code, for example in b.__iter__.next() from obtaining a reference to x via any number of mechanisms and then altering its attribute dict.

You wouldn’t even have to resort to reflection - the local variable x could just be a reference to an object which is also reachable in a global scope. And what variables there are in the global scope can be modified on the fly by any code so it wouldn’t be enough to do an escape-analysis at compile time.

Frankly there are just too many avenues in python for this sort of jiggery-pokery to take place. And these avenues are kind of the point of python.

Locals are Fast In Python

OK, that’s interesting, but what if we just explicitly apply the above optimisation?

def partition(pred, it): # scara
    ts = []
    fs = []
    t = ts.append
    f = fs.append
    for item in it:
        if pred(item):
            t(item)
        else:
            f(item)
    return fs, ts

You can see from the bytecode that this has reduced the instruction path-length as we expected:

 28           0 BUILD_LIST               0
              2 STORE_FAST               2 (ts)

 29           4 BUILD_LIST               0
              6 STORE_FAST               3 (fs)

 30           8 LOAD_FAST                2 (ts)
             10 LOAD_ATTR                0 (append)
             12 STORE_FAST               4 (t)

 31          14 LOAD_FAST                3 (fs)
             16 LOAD_ATTR                0 (append)
             18 STORE_FAST               5 (f)

 32          20 LOAD_FAST                1 (it)
             22 GET_ITER
        >>   24 FOR_ITER                30 (to 56)
             26 STORE_FAST               6 (item)

 33          28 LOAD_FAST                0 (pred)
             30 LOAD_FAST                6 (item)
             32 CALL_FUNCTION            1
             34 POP_JUMP_IF_FALSE       46

 34          36 LOAD_FAST                4 (t)
             38 LOAD_FAST                6 (item)
             40 CALL_FUNCTION            1
             42 POP_TOP
             44 JUMP_ABSOLUTE           24

 36     >>   46 LOAD_FAST                5 (f)
             48 LOAD_FAST                6 (item)
             50 CALL_FUNCTION            1
             52 POP_TOP
             54 JUMP_ABSOLUTE           24

 37     >>   56 LOAD_FAST                3 (fs)
             58 LOAD_FAST                2 (ts)
             60 BUILD_TUPLE              2
             62 RETURN_VALUE

The inner loop takes place between locations 24 and 48. It’s 16 instructions long. And if we look at the instruction path length for the loop, it’s now 11 instructions long. One shorter than before. Of course, it’s that missing LOAD_ATTR instruction which has been hoisted out of the loop in to the function preamble.

A Bit More Squeezing

I couldn’t figure out a way to get the inner loop any tighter but I did find a way to make the code more compact by removing duplicated code on both sides of the branch. It led to a tiny improvement in performance. Here it is:

def partition(pred, it): # scara2
    ts = []
    fs = []
    t = ts.append
    f = fs.append
    for item in it:
        (t if pred(item) else f)(item)
    return fs, ts

 40           0 BUILD_LIST               0
              2 STORE_FAST               2 (ts)

 41           4 BUILD_LIST               0
              6 STORE_FAST               3 (fs)

 42           8 LOAD_FAST                2 (ts)
             10 LOAD_ATTR                0 (append)
             12 STORE_FAST               4 (t)

 43          14 LOAD_FAST                3 (fs)
             16 LOAD_ATTR                0 (append)
             18 STORE_FAST               5 (f)

 44          20 LOAD_FAST                1 (it)
             22 GET_ITER
        >>   24 FOR_ITER                24 (to 50)
             26 STORE_FAST               6 (item)

 45          28 LOAD_FAST                0 (pred)
             30 LOAD_FAST                6 (item)
             32 CALL_FUNCTION            1
             34 POP_JUMP_IF_FALSE       40
             36 LOAD_FAST                4 (t)
             38 JUMP_FORWARD             2 (to 42)
        >>   40 LOAD_FAST                5 (f)
        >>   42 LOAD_FAST                6 (item)
             44 CALL_FUNCTION            1
             46 POP_TOP
             48 JUMP_ABSOLUTE           24

 46     >>   50 LOAD_FAST                3 (fs)
             52 LOAD_FAST                2 (ts)
             54 BUILD_TUPLE              2
             56 RETURN_VALUE

Now we’re down to a 13 instruction inner loop with an 11 instruction path-length.

Benchmarks

For benchmarking I loaded a dictionary of some half a million words, created a set of all the words that begin with “s”, and then partitioned the dictionary based on membership of the “s” set.

Tests were performed on an i7-6600U laptop with python 3.9.0 and PYTHONHASHSEED=0

dic = tuple(Path('/usr/share/dict/words').read_text().splitlines())
needles = frozenset((word for word in dic if word.startswith('s')))
for func_name in ('stdlib', 'mariy', 'trevor', 'byers', 'scara', 'scara2'):
    print(func_name, timeit(
        f'{func_name}(pred, dic)',
        setup=f'from __main__ import dic, pred, {func_name}',
        number=32,
    ))

Variant	Time for 32 iterations (seconds)
stdlib	3.5664224450010806
mariy	3.532839963911101
trevor	2.5254638858605176
byers	2.356995061971247
scara	2.1279552809428424
scara2	2.087127824081108

A Parting Shot

Here it is with all the mypy --strict typing goodness added:

from typing import List, Any, Callable, Iterable, TypeVar, Tuple

T = TypeVar('T')

def partition(pred: Callable[[T], bool], it: Iterable[T]) \
        -> Tuple[List[T], List[T]]:
    ts: List[T] = []
    fs: List[T] = []
    t = ts.append
    f = fs.append
    for item in it:
        (t if pred(item) else f)(item)
    return fs, ts

Arbitrary Remote Code Execution is (Still) Arbitrary Remote Code Execution

2020-02-08T01:14:00+00:00

Long gone are the days when servers were like little pets. You logged in to them, you stroked them, you ran little commands on them. Probably some commands you copied and pasted out of stackoverflow or that you found in some google search results.

Then when the server died, you had a little funeral, buried it in a little pet cemetery, and then tried to remember how the hell you had set it up.

But that was fine, you were in grief, you didn’t want another pet to be identical to your dead pet anyway.

That is, Unless you were trying to deliver consistent, performant, and uninterrupted service to colleagues, customers, or the like.

Cattle, not Pets

In that case you’d want something a bit more repeatable. A bit more automated. A bit more like a production-line eviscerating pigs for the manufacture of sausage.

First there were configuration management systems, like:

1993: CFEngine
2000: Spooner (proprietary, developed by yours truly) :)
2005: Puppet
2009: Chef
2012: Ansible

These tools allowed you to represent the configuration or desired state of a server as code or data but, in either case, as a version-controlled repository subject to the quality-oriented practices which began to be adopted by programmers in the early decades of the 21st century.

Later, in 2013, Docker came along. The idea behind docker was to use containers as a way to build or deploy software in an industrialised fashion, like the afore-mentioned sausage factory. The more inspirational, but frankly less tasty, metaphor here would be the containerisation of bulk cargo. Before this idea took hold, containers were thought of as more to do with the containment or confinement of running software. They were sometimes called jails and still widely viewed as being fancy chroots.

In combining the underlying technology of namespaces, control-groups, jails, containers with the philosophy and concepts of infrastructure-as-code. Docker, and systems like it, revolutionised how software services were delivered. By now these, and related, techniques are ubiquitous in the data-centre.

Which is great, because now all of the insecure and insane practices get written down and permanently archived in git repositories instead of being a secret burden of guilt and shame carried on the shoulders of an entire class of people: systems administrators.

Now the blame can be diffused among software developers of all hues and stripes and we can call it a cultural problem :)

Building and Bootstrapping Infrastructure Now.

Well, a container really is just a fancy chroot. Whereas chroot changes the processes view of the VFS namespace by setting the root to some directory, thereby restricting its view of the filesystem to a subtree of that directory. Containers are processes which have their own private, possibly unique, set of mounts.

However, we still need to populate the mounted directories or filesystems with the software that’s going to run!

There are any number of ways of doing this.

SnapCraft

Snapcraft uses a YAML file which defines sources to be downloaded, which will then be compiled and installed on top of a base OS layer which contains compilers, a package manager, etc.

Snapcraft provides security measures, such as the ability to check source code downloads against checksums with the source-checksum feature and the ability to download code over https.

However, if you just download code from the internet over http, then you’re just an ettercap or a dns poisoning away from being fed exploit code which will automatically be run as part of the snap build process.

The snapcraft tour, part of the getting started guide, suffered from this problem, which was reported in Oct 2016 and was fixed in May 2017.

Which is great. We shouldn’t be teaching a new generation of programmmers to automatically trust and execute arbitrary remote code.

LXC Templates : CVE-2017-18641

LXC shipped a series of templates which are scripts which initialise the container root filesystem.

The centos and fedora templates relied on yum being installed on the host. The hosts yum would be used to download RPMs and install them in to the rootfs.

The RPMs were being downloaded over http, which could be okay, since RPMs are signed. However yum was being invoked with --nogpgcheck which disabled this feature. The upshot being that anyone using the template is also an ettercap or a hacked mirror or untrustworthy proxy away from having arbitrary code executed as root.

This was reported on Feb 3 2017. And it turned out after a quick investigation that the problem applied to around a dozen of the template scripts.

Some were fixed in short order. But there were so many cases of this, all in different ad-hoc scripts, by different authors, that it became “a bit of a mess.”

By Feb 2 2018 the LXC team had started work on distrobuilder which had support for https and gpg from the outset and would be difficult to get wrong.

By LXC 3 the templates system had been removed to a historical repo and the status quo is now that images are built securely and signed on trusted infrastructure and then sent to users over https and with a signature which is checked by lxc-download.

This is a fine example of the concerns which go in to building a software distribution mechanism in which trust can be established.

Insecurely Downloaded Remote Code is, by Definition, Arbitrary.

And If it’s Arbitrary then You Have No Reason to Trust it.

So Why Would You Want to Execute it?

It’s certainly tempting, when writing a Dockerfile, to imagine that everything is executing in some sort of secure isolation layer and that, because of this, security is irrelevant.

For now, let’s ignore the historical record security vulnerabilities in container isolation, and the entire available attack surface of the kernel, and the fact that namespaces and control groups are not purpose-built security isolation systems. And the fact that even if they were such, then with all their flexibility and configurability there would bound to be many possible insecure configurations. Let’s ignore all of that for now.

Usually, if you’re creating a container, you want to be able to trust the code inside it with whatever else is in that container. So even if an attacker can’t break out to the host, you won’t want them to have full control over the container because they may be able to deface your website, defraud your customers, introduce backdoors in to your software builds, etc.

And if the attacker has corrupted the image at container build time. Then you don’t even have the defence that the container instances are cattle that can be shot in the head with a bolt-gun and replaced with a fresh new lump of meat, cloned from the same DNA.

So are these practices common?

From a quick search on github I was able to find 23,000 instances of “wget http://…” in Dockerfiles.

By randomly clicking them you can see that most of these (100% of the ones I looked at anyway) absolutely are downloading arbitrary code and then running it.

I was also able to find a further 4,000 instances for curl, but some of these looked legitimate.

With a bit of regex trickery one could probably find many more instances.

GitHub really is a repository of endless trivial security vulnerabilities.

A Brief Message from the Cast

While software vendors take vulnerabilities seriously when they are discovered. And act to remediate them in a timely and responsible fashion. We probably need to be doing a little bit more, as a community, for the typical users of these things to drill in to them the dangers involved.

It’s never going to be comfortable for purveyors of band-saws to be always pointing out that band-saws can irreparably alter the relation between a user’s fingers and said user’s hands.

That’s just not the fun stuff, compared to all the cool things you can make and do with power-tools.

However, when we see how widespread some of these unsafe practices are, we might want to think about what sort of technical or social interventions might work to improve the situation.

Python socket servers can drop received packets on exit

2019-06-16T01:14:00+00:00

Let’s say we have a datagram server written in python using the shiny new asyncio module:

import socket
import asyncio
import os

class PacketCounter(asyncio.DatagramProtocol):
    def __init__(self):
        self.counter = 0
    def datagram_received(self, data, addr):
        self.counter += 1

class Writer:
    def __init__(self, socket_path, nr_msgs):
        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM, 0)
        self.sock.connect(socket_path)
        self.sock.setblocking(False)
        self.sent_msgs = 0
        self.nr_msgs = nr_msgs
        loop.add_writer(self.sock.fileno(), self.writable)
        self.wait = loop.create_future()

    def writable(self):
        while self.sent_msgs < self.nr_msgs:
            try:
                self.sock.send(b'Hello')
                self.sent_msgs += 1
            except BlockingIOError:
                return
        if self.sent_msgs >= self.nr_msgs:
            loop.remove_writer(self.sock.fileno())
            self.wait.set_result(None)

loop = asyncio.get_event_loop()

socket_path = 'hello'

# Create the server
try:
    os.unlink(socket_path)
except FileNotFoundError:
    pass
proto = PacketCounter()
t = loop.create_datagram_endpoint(lambda :proto,
                                  family=socket.AF_UNIX,
                                  local_addr=socket_path)
transport, _ = loop.run_until_complete(t)

# Create the client and run it
sender = Writer(socket_path, 1000)
loop.run_until_complete(sender.wait)
transport.close()
loop.close()

print('tx', sender.sent_msgs)
print('rx', proto.counter)

If we run it, we get:

tx 1000
rx 841

Hrm…

What’s going on?

When the kernel receives a packet it gets placed in to the relevant socket’s receive buffer, the socket becomes readable potentially causing a wakeup event in poll(), epoll_wait(), or a synchronous read() or recv().

In the case of a datagram socket, a recv() will be woken up once for each received datagram. This is how the message boundaries are preserved.

So let’s say the kernel receives 3 messages for a datagram socket. The application will need to call recv() 3 times to receive all of those. Or equivalently, sleep in poll() 3 times and do a non-blocking recv() each time.

In any case, when it tries to read for a fourth time, it will either sleep if the socket is a blocking socket (the default), or it will return EAGAIN error (or raise BlockingIOError in python). There is no more data remaining in the kernel’s buffer, so the socket is not ready.

Now, in python’s selector event loop (the default for UNIX-like operating systems). When socket becomes readable, python wakes up from epoll_wait() (for example), eventually calls _SelectorDatagramTransport._read_ready which performs the sock.recv() and then calls protocol.datagram_received() That will process that one single datagram. After that we’ll go through the other selector events and then go back to sleep in epoll_wait().

However, since the sender has finished it’s work loop.run_until_complete() exits and now we try and close everything down.

But even though I gave the socket server every possible chance to grab everything in the kernel’s socket buffer (transport.close(), loop.close()). Those remaining packets that could have finished a perilous journey accross the world to get in to my kernel’s receive buffer just get dropped to the floor like so many crumpled up cigarette boxes.

Why this sucks

First of all, this confused me because I’m used to event frameworks in C, using epoll, where the mainloop doesn’t finish an iteration until all event sources have hit EAGAIN, thereby draining the kernel’s socket buffer of received data before exiting.

In python asyncio, however, you need to be totally explicit if you want this behaviour. But that’s okay, right? These are all valid design choices, after all.

I’m not so sure. You see, in BaseSelectorEventLoop._read_from_self() it looks like the right kind of behaviour happens when the self-pipe becomes readable:

def _read_from_self(self):
while True:
    try:
	data = self._ssock.recv(4096)
	if not data:
	    break
	self._process_self_data(data)
    except InterruptedError:
	continue
    except BlockingIOError:
	break

But if I want this sort of behaviour for my datagram server I can’t easily mess around with _SelectorDatagramTransport._read_ready and such-like.

I have to use some kind of workaround like this:

# ... 8< ...
# Create the client and run it
sender = Writer(socket_path, 1000)
loop.run_until_complete(sender.wait)

# Now, drain the kernel socket-buffer
while True:
    try:
        buf = sock.recv(64 * 1024)
    except BlockingIOError:
        break
    proto.datagram_received(buf, None)
transport.close()

loop.close()

print('tx', sender.sent_msgs)
print('rx', proto.counter)

But this is going to get seriously annoying if you have multiple listening sockets all over your program.

On the other hand, if I am worried that my program won’t exit because I’ll be spending forever servicing a never-ending stream of datagrams, I think that that’s a much easier problem to solve. You just close the socket, then there’s no way to keep receiving stuff.

A solution

Why not use edge-triggered epoll and implement the edge-triggered behaviour under the hood (re-try callbacks until BlockingIOError) without the user having to know about it? It’s going to result in way fewer system calls, and way fewer gotchas like this. And if you want the old behaviour it’s easy to just break out of the loop immediately by calling loop.stop().

There will be concerns about “starvation” but starvation can be easily avoided by putting ready file-descriptors in to a “ready queue” and looping over the list calling each callback in round-robin (hell, priority-queue for all I care) fashion. Then you can just epoll_wait(..., 0) after some number of iterations has been done but the ready-queue is still non-empty. That will just get you any new fd’s which became writable in the meantime and you can add those to the end of the “ready queue.”

Seriously guys… come on :)

Abusing the CPU’s adder circuits

2019-06-15T01:14:00+00:00

Have you ever been asked the interview question “how do you count the number of bits set in an integer?”

A smart-ass like myself might answer something like:

unsigned int count_set_bits(unsigned int x)
{
	return __builtin_popcount(x);
}

Which, on x86_64, produces:

	xor	eax, eax
	popcnt	eax, edi
	ret

The interviewer, of course, doesn’t like that. He wants to see you come up with an algorithm. So you come up with something like this:

unsigned int count_set_bits(uint32_t x)
{
	unsigned int i, ret;
	for(ret = i = 0; i < 32; i++) {
		ret += !!(x & (1U << i));
	}
	return ret;
}

Which gives you something like:

	xor	r8d, r8d
	xor	eax, eax
	mov	ecx, 1
	.p2align 4,,10
	.p2align 3
.L4:
	shlx	edx, ecx, eax
	test	edx, edi
	setne	dl
	movzx	edx, dl
	inc	eax
	add	r8d, edx
	cmp	eax, 32
	jne	.L4
	mov	eax, r8d
	ret

Which is, of course, dreadful. But the reason the interviewer likes this is because it sets up the next question: “how can we optimize this?” Or if you want to be more circumspect: “what if we only expect one or two bits to be set?” or “what if our data is very sparse?”

Realistically though, you either know about “Kernighan’s trick”, or, you don’t. It goes something like this:

unsigned int count_set_bits(unsigned int x)
{
	unsigned int ret;
	for(ret = 0; x; x &= (x - 1))
		ret++;
	return ret;
}

Which, at least for an Intel weenie such as myself, was a pointless exercise because gcc compiles that to:

	xor	eax, eax
	popcnt	eax, edi
	mov	edx, 0
	test	edi, edi
	cmove	eax, edx
	ret

Which looks like a terrible code-generation bug in gcc. But hey, at least clang does the right thing here:

	popcnt	eax, edi
	ret

Anyway, I digress. Did you notice the x &= (x - 1) part? That is bitwise woo-woo magic to unset the rightmost bit in x. It’s a neat trick, and if you know it you’ll ace this part of the job interview.

But later, when you’re at home, alone, contemplating the empty meaninglessness of the universe, you might ask yourself “but why does it work?”

Two’s complement

Let’s forget the & part and just focus on what x - 1 is doing for now.

One thing you’re going to need to know about is two’s complement arithmetic.

I won’t bore you by recapitulating the details but we should do well to remember the following equations:

\[\begin{align} x - y & = x + -y \\ -x & = \tilde x + 1 \\ \end{align}\]

Therefore:

\[\begin{align} x - 1 & = x + -1 \\ &= x + 1111... \end{align}\]

So the magic rune is just adding all one’s to our original value.

Half adder

So here’s the part where we get in to how addition really works. You probably have a good handle on this already from your school days, but let’s refresh.

The simplest case is if we’re just adding together a couple of 1-bit numbers.

Let’s write out the truth table for that:

x	y	result
0	0	0
0	1	1
1	0	1
1	1	0… oh yeah, we need to carry…

Let’s try again:

x	y	result	carry
0	0	0	0
0	1	1	0
1	0	1	0
1	1	0	1

Right, if we’re going to expand this to add numbers with more than one bit, we’re going to need to do something about this carry output.

Full adder

Let’s call the last adder with 2 inputs, and 2 outputs, the “half-adder.”

The full adder is going to have 3 inputs: the 2 operands, and then a carry-in which is going to be connected to the next least significant bit’s carry-out.

x	y	carry-in	result	carry-out
0	0	0	0	0
0	1	0	1	0
1	0	0	1	0
1	1	0	0	1
0	0	1	1	0
0	1	1	0	1
1	0	1	0	1
1	1	1	1	1

So you can see in this example of a 4-bit adder that there’s this chain of carry inputs going from right to left:

Thanks to Colin M.L. Burnett for the excellent diagram.

Putting it all together

The basic outline of the algorithm is that we start with an input, x. And as long as it isn’t zero, we unset the least significant bit, and increment the result.

So if the input is 1000_1100, the loop will iterate 3 times.

x &= (x - 1) is an operation which clears the least-significant set bit in the input, x.

Since we’re going to do x &= mask, that must mean that mask has a zero in the position we want to zero out. And it can’t have any zeroes where the input has a one.

So we abuse the carry-chain to create such a mask. By adding 1111_1111 to the input, we’re essentially ensuring that the carry signal is not asserted until we hit the first one bit, but after that it will be asserted for all the rest of the bits (travelling from right to left).

If we extract the relevant parts of the adder truth table we can see how the addition operation will produce the mask that we need:

x	y	cin	result	cout	explanation
0	1	0	1	0	Before the first bit, produce ones
1	1	0	0	1	For the first bit, produce a zero and set the carry going
0	1	1	0	1	now the result is zero if x was 0
1	1	1	1	1	or 1 if x was one

What that gives us is a number which is (from right to left):

all ones (but that’s okay because corresponding input bits are zeroes)
zero for the first one bit (which ensures that it will be cleared)
and then equal to the input after that (leaving all higher bits unchanged)

SQL-Like Sorting in Python

2019-03-16T14:20:00+00:00

Sorting things in python can often be a pain point. You need to be familiar with the decorate-sort-undecorate paradigm. Also known as the Schwartzian transform. In python3 this technique replaced the old system of comparator functions. Lots of words have been expended discussing this in the quite excellent python documentation so I won’t try and duplicate any of that here.

For what it’s worth, I actually agree with the python implementors that this is the better, less error-prone approach to the problem. And it can have some performance benefits too.

But what if you crave simpler times? Flash back to when Eric Clapton’s cover of “I shot the sheriff” was blaring out of everyone’s 8-tracks. SQL gives us a pretty nice and intuitive formalisation for defining orderings over bags of tuples. And iterable sequences of objects in python can be considered exactly that.

Let’s get the preliminaries out of the way. This is what the use-case looks like:

from types import SimpleNamespace
from enum import Enum
class Order(Enum):
    ASC = 0
    DESC = 1

keyfunc = sqlordering(('x', Order.ASC), ('y', Order.DESC))

input = [
    SimpleNamespace(x=999, y='aardvark'),
    SimpleNamespace(x=999, y='xylophone'),
    SimpleNamespace(x=111, y='xylophone'),
    SimpleNamespace(x=111, y='aardvark'),
    # etc..
]

for item in sorted(input, key=keyfunc):
    print(item)
# SimpleNamespace(x=111, y='xylophone')
# SimpleNamespace(x=111, y='aardvark')
# SimpleNamespace(x=999, y='xylophone')
# SimpleNamespace(x=999, y='aardvark')

The key function

Let’s work backwards from the key function required by list.sort(). The simplest way to produce a total ordering is to define a class whose constructor takes one argument, the item that we want to compare/sort. That class should keep a reference to the original object and then define the six rich comparison methods.

We’ll use the functools.total_ordering decorator to fill in a lot of boilerplate for us. This way we’ll only need to define an __lt__ and an __eq__ method in our class and the decorator will fill in the rest for us. All those other methods can be defined in terms of just those two (e.g. __ne__ is equivalent to not __eq__).

Regardless, it looks like sorted and list.sort use only the __lt__ comparison so those methods ought never be called and there should be no performance hit. But we may as well define them just because there’s no such thing as a guarantee in life. But that’s another topic, and you can read about my failures and disappointments in life in my upcoming memoir.

def sqlordering(*spec):
    @total_ordering
    class K:
        __slots__ = ['_obj']
        __hash__ = None
        def __init__(self, obj):
            self._obj = obj
        def __lt__(self, other):
	    # Implement this
	    pass
        def __eq__(self, other):
	    # Implement this
	    pass
    return K

How to define lt and eq

The first thing we need to do is iterate over the sort specification. For any field sorted ASCending we’ll use operator.lt (less-than) and for any field sorted DESCending we’ll use the opposite: operator.ge (greater-than-or-equal).

Then we’ll need to use partial application to combine the attribute lookups with the operator.

For example, to create a function f which takes two objects and returns True if attribute x on the first argument is less than attribute x on the second argument, we can do:

from functools import partial
# a is attribute name, a string
# c is comparison operator, a callable
# x,y will remain free paramaters and can be any type
cmp_func = lambda a,c,x,y:c(getattr(x,a),getattr(y,a))
f = partial(cmp_func, 'x', operator.lt)

Now, with that, we need to create a higher order function which combines this sequence of functions to create the two operators for our total ordering.

For the equals operator it’s straightforward, two items are equal if and only if all attributes are equal. So we just iterate over the sequence of equality functions applying them as we go. If any of them fail, the comparison fails.

For the less-than operator it’s a little bit more involved. Recall the semantics of SQL sorts. You can find that in section 8.2 “<comparison predicate>” of ISO/IEC 9075-2:2016, general rules, paragraph 1, clause h:

The relative position of row P is before row Q if PV_n precedes QV_n for some n, 1 (one) ≤ n ≤ N, and PV_i is not distinct from QV_i for all i < n.

In other words, if two rows are equal in the first comparison key, then we proceed to ordering by the second comparison key, and so on.

So with that we can go ahead and complete the implementation. Here it is:

from typing import Iterable, Tuple
from functools import total_ordering, partial
from operator import lt, ge, eq

def _cmp_func(*args):
    spec : Iterable[Tuple[str, Order]] = args
    cmp_func = lambda a,c,x,y:c(getattr(x,a),getattr(y,a))
    for attr, sense in spec:
        c_eq = partial(cmp_func, attr, eq)
        if sense == Order.ASC:
            c_lt = partial(cmp_func, attr, lt)
        elif sense == Order.DESC:
            c_lt = partial(cmp_func, attr, ge)
        else:
            raise ValueError
        yield (c_lt, c_eq)

def sqlordering(*spec):
    funcs = tuple(_cmp_func(*spec))

    @total_ordering
    class K:
        __slots__ = ['_obj']
        __hash__ = None
        def __init__(self, obj):
            self._obj = obj
        def __lt__(self, other):
            for ltcmp, eqcmp in funcs:
                if ltcmp(self._obj, other._obj):
                    return True
                elif not eqcmp(self._obj, other._obj):
                    return False
            return False
        def __eq__(self, other):
            for ltcmp, eqcmp in funcs:
                if not eqcmp(self._obj, other._obj):
                    return False
            return True
    return K

Item-at-a-time Reduce Functions in Python

2019-03-09T14:28:47+00:00

If you need to write a Reduce function in python, there’s a number of ways of doing it. I’m going to assume you already know what that is and leap right in. Suffice to say, you can think of that as something like an aggregate function in an SQL query.

Perhaps the most obvious would be to use functools.reduce.

from functools import reduce
from operator import add

reduce(add, range(20))

But in this case you would need the entire iterable to be ready at the time of calling reduce. What if you want to call the reduce one step at a time? This would be particularly important, for example, if you were receiving the items over a network connection or reading them from a huge file.

With Generators

Another way might be to use a generator, but that’s not without it’s problems either:

from operator import add

def genreduce(init_val, func):
    val = init_val
    yield init_val
    while True:
        nxt = yield val
        val = func(val, nxt)

def online_reduce(func, init_val, gen):
    state = genreduce(init_val, func)
    init_val = next(state)
    for item in gen:
        init_val = state.send(item)
    return init_val

So now things have got pretty ponderous. And the two things we want to specify here are quite cumbersome to carry around - the function defining the reduction and it’s initial value.

With Function Attributes

What I’m really after is a concise way to specify the callees - the reduce functions themselves. And an intuitive pythonic protocol for a caller to use them.

Luckily, in Python, we can just add attributes to functions.

def foldsum(prev_val, val):
    return prev_val + val
foldsum.init_val = 0

def online_reduce(func, gen):
    val = func.init_val
    for item in gen:
        val = func(val, item)
    return val

This is nice, it lets us combine the function and the initial-value without resorting to defining some new class, or just using a tuple like (add, 0) which is essentially untyped and can get confusing.

It does make type checking blow up though.

With a Decorator

Lets try again but make this a function decorator. We want to be able to decorate our own functions as well as things like operator.add, so we’ll need an extra level of indirection so we don’t start adding attributes on to members of the operator module!

When it comes to typing, we want the caller to be able to access func.init_val without causing a type error. We can define a custom protocol for that using the typing_extensions module.

But also, our actual decorator needs to not blow up mypy and that gets a little bit tricky. I couldn’t figure out how to avoid using a cast.

With all that said, here’s all the tedious boilerplate for the decorator:

from typing import Callable,Any,cast
from typing_extensions import Protocol
from functools import wraps

class ReduceFunc(Protocol):
    init_val : Any
    def __call__(prev_val, val):
        raise NotImplementedError

def reducer(parm : Any):
    def decorator(func) -> ReduceFunc:
        @wraps(func)
        def wrapper(a, b):
            return func(a, b)
        rf = cast(ReduceFunc, wrapper)
        rf.init_val = parm
        return rf
    return decorator

__all__ = (
    'reducer'
)

And here’s two ways of defining a reduce function. First for a user-defined function, secondly when wrapping a function from somewhere else like the operator module.

@reducer(0)
def foldsum(prev_val, val):
    return prev_val + val

from operator import add
foldsum = reducer(0)(add)

Getting Bitmasks from SSE Vector Comparisons

2019-03-08T09:49:47+00:00

These days the single-threaded performance of CPUs just isn’t advancing as quickly as it did in my salad days of the late 90s and early 2000s. Personally I think it’s God’s revenge for people putting pineapple on pizza. Regardless of the causes, I find it to be increasingly the case that if you aren’t using SIMD in the performance critical areas of your software, then you are leaving a lot of performance on the floor.

Let’s say that you have an array of uint16_t’s and you want to go through them all looking for all instances of a given number. In plain-old-C you would do something like:

void my_pipeline_is_as_empty_as_my_soul(const unsigned count,
					const uint16_t vector[static count])
{
	unsigned int i;
	for(i = 0; i < count; i++) {
		if ( vector[i] == 0x1234 ) {
			/* do the thing */
		}
	}
}

But you’ve been chilling listening to “smooth” by Carlos Santana, and you’ve heard of this hip new technology called SSE2. It allows you to do EIGHT of these comparisons in a single bound!

So you include the relevant header and you start working on an inner-loop which handles 8 items at a time. We’ll worry about any trailing data later. You’ll use _mm_load_si128 to just slurp an array of 8 elements in to one of these new wide-boy registers:

#include <emmintrin.h>
uint16_t in16[8] = {
	/* 0 */ 0x1234,
	/* 1 */ 0x4567,
	/* 2 */ 0x1234,
	/* 3 */ 0x1234,
	/* 4 */ 0x1234,
	/* 5 */ 0,
	/* 6 */ 0x1212,
	/* 7 */ 0x3434,

};
__m128i h = _mm_load_si128((__m128i *)in16)
// 1234 4567 1234 1234 1234 0000 1212 3434

And, as before, we are looking for all instances of 0x1234 so we do a comparison using the _mm_cmpeq_epi16 intrinsic:

__m128i n = _mm_set1_epi16(0x1234);
// 1234 1234 1234 1234 1234 1234 1234 1234
__m128i r = _mm_cmpeq_epi16(n, h);
// ffff 0000 ffff ffff ffff 0000 0000 0000

Astute readers will notice that these equality tests can be achieved with and-ing or xor-ing, and that’s correct. But the topic of this post is the comparison operators. These become really useful when you want to do range comparisons using greater-than and/or less-than or things like that.

Anyway, this is all wonderful but what am I going to do with this weird two-byte boolean-like thing?

I now have a vector of uint16_t’s which are set to all 1s for matching items and all 0s for non-matching items. We could just break these out in a loop but that would waste all the effort we’ve put in to avoid branching and looping by doing this with SIMD in the first place.

It would be nice if we could get the results in a bitmask, so we can do other set-wise operations on all 8 elements at once. Or use __builtin_ctzl and __builtin_clz respectively to find the first/last set bits.

The solution

How we’re going to do this is by using _mm_packs_epi16 to collapse the 16bit values down to 8bit values and then _mm_movemask_epi8 to extract the mask.

__m128i p = _mm_packs_epi16(cmp_res, cmp_res);
// ff 00 ff ff ff 00 00 00 (x2)
int mask = _mm_movemask_epi8(p) & 0x7f;
// 0x1d (or 00011101)

Now, this might require a bit of unpacking - if you will excuse the pun. At first glance _mm_packs_epi16 doesn’t seem an obvious choice. Intel describes it as:

Convert packed 16-bit integers from a and b to packed 8-bit integers using signed saturation, and store the results in dst.

To put this in plain english, what happens is that, for each 16 bit integer in our input vector we split it down in to two 8-bit integers and add them together. But, instead of wrapping back to zero when we overflow, the values get ‘stuck’ at the highest possible value of 0xff (ie. saturated addition). The resulting 8-bit integer is appended to the result vector. A C implementation would look something like this:

uint8_t in[32]; // this is the concatenation of the two 128bit operands
uint8_t out[16]; // output operand

for(i = 0; i < 16; i++) {
	unsigned int one, two, res;

	/* Load the input */
	one = in[i * 2 + 0];
	two = in[i * 2 + 1];	

	/* Saturated addition */
	res = one + two;
	if ( res > 0xff ) {
		res = 0xff;
	}

	/* Store the result */
	out[i] = res;
}

To avoid adding a dependency on any other registers we use the same input register for both operands. This leaves us with two identical copies of the desired output in the upper and lower 8 lanes of the output register respectively. That’s why after converting to a bitmask with _mm_movemask_epi8, we mask out the upper 8 bits to ensure that they’re always zero.

This is, of course, a little wasteful. If we’re looping through a huge array would just do two lots of comparisons so we can fill two registers full of results and then do a single _mm_packs_epi16 to generate a 16 bit mask.

What about with 256 and 512 bit vectors?

The 256 bit version looks much the same. It produces a 16-bit bitmask at the end and uses immintrin.h and the corresponding SSE3 versions of each operation.

#include <immintrin.h>
__m256i h = _mm256_load_si256((__m256i *)ptr);
__m256i n = _mm256_set1_epi16(0x1234);
__m256i r = _mm256_cmpeq_epi16(n, h);
__m256i p = _mm256_packs_epi16(r, r);
int mask = _mm256_movemask_epi8(p) & 0xffff;

For AVX-512 you needn’t go through the rigmarole because of the in-built support for mask registers.

__m512i h = _mm512_load_si512((const __m512i *)ptr);
__m512i n = _mm512_set1_epi16(0x1234);
__mmask32 mask  = _mm512_cmpeq_epi16_mask(n, h);

Easy! And it’s nice and fast. Looks like you can run comparisons on an entire cache line, 32 uint16_t’s at a time, with only 1-cycle of latency. And you can run two at a time in parallel on a single core. Pretty neat.

Tagliatelle al sugo di cipolle e pancetta

2018-12-05T14:00:00+00:00

That’s big flat spaghettis with bacon and onion sauce to you and me. It’s a bit less dense than carbonara but just as nice. It’s also a good dish for learning the basic techniques of italian cuisine which you can apply in a dozen other other dishes by substiting one or two ingredients.

You will need:

Some pasta, potable water, salt worthy of its salt.
Some pancetta, I suppose bacon will do, no need to be a snob about it.
Some cheese, parmigiano reggiano or peccorino romano. I would avoid the fake stuff here and look for the DOP symbol. If you’re in the EU this won’t be a problem.
Freshly ground black pepper. Which, in italian cuisine, is a spice to be taken seriously - not some black dust added as an afterthought. You want to be able to taste it.
Some white wine. My tastes are cheap so any old plonk will do.
A non-non-stick pan. Made of stainless steel by steely-eyed german men.

For two people, 250g of pasta, 50g of cheese and 100-150g of meat should be a feast.

Start by cutting the pancetta in to cubes. Fry them off in a pan until the fat has rendered and they start to brown the bottom of the pan. Grind some black pepper in to the fat and meat. Add in a finely chopped onion and also fry until brown.

Deglaze the pan with a splash of white wine and cook off the alcohol for a minute. There should still be a little fond left on the pan after this. Add a ladleful or two of the pasta water, which should be nice and starchy by this point and make sure to scrape up all the fond from the bottom of the pan.

Grate some parmegiano reggiano cheese in to a bowl and add a little pasta water there too. Fold the cheese and water in to the consistency of a thick paste. This will make it easier to incorporate in to the pasta and sauce without going all clumpy and rubbery.

By this point the pasta should be pretty al dente, a minute or two from done. Dump the pasta in to the frying pan and stir them around making sure to get the sauce coated on the noodles. Fold in the cheese mixture too. It should all emulsify to form a thick and creamy sauce that coats all the pasta.

You will need to taste it at this point. You should have salted the pasta water well and it will now be reducing and getting saltier. Also the cheese and pancetta is salty so you need to be careful when you bring these elements together. Salt cannot be removed once added. Well, you could, but the food won’t be as texturally satisfying after being pumped through an osmosis filter. So what I do is to use less salt in the pasta water than I usually would when cooking this way. Then I can add salt and cheese and adjust the seasoning here to make sure I don’t end up with bland mush.

Once the seasoning is right and the pasta is cooked to al dente, you are ready to serve. Twist the pasta up on a fork or tongs and place on the plate. Add some more black pepper, and sprinkle some grated cheese on top. If you want to be even more fancy you could have set aside half of the meat and arranged it decoratively before luxuriously draping the individual strands of grated cheese atop the dish.

But personally, I like to just eat it right from the pan with a fork while wearing boxer shorts and a white cotton vest.

From here you can sub the onions for eggs, and get carbonara. Or you could skip the meat and onions and use peccorino cheese and end up with caccio e pepe. You could add a small amount of tomatoes for a more fruity variant. Subtract onions from that and go for bucatini all’amatriciana. You could replace the pancetta with tuna and perhaps the wine for parsley and lemon juice and end up with tonno e cippole.

Whatever you chose, at least you can impress your friends and enjoy restuarants a lot less by knowing how to cook pasta properly.