Link Search Menu Expand Document

Week 10: Networking

Homework

Computer systems are complicated. It’s somewhat magical that you can send a message from your phone and have a relative halfway across the world receive it! But how does it all work? Today, we’ll be exploring the magic of computer networking in all its glory.

This lecture (like all others) is a very informal treatment of basic networking, taken mostly from the 168 textbook. It won’t be enough for you to learn the nitty-gritty details of things like queueing or delay, or small details like physical implementation. Our goal is to provide you with enough understanding to cut your teeth on tools in the networking world.

Intro

Generally, we all have this vague idea that computers work according to something like this:

[your phone] <---> ??? <---> [server] <---> ??? <---> [friend's phone]

Keeping this mental model in mind, let’s first talk about what goes across the network before we talk about what exactly the network looks like.

Messages, Packets, and Packet Transfer

At the very basic level, we want to route messages between two devices on a network. These messages are broken down into chunks called packets, which are the actual things routed across the internet. To visualize this, imagine that you want to send a message to your friend, having only a postal system that lets you send messages with a 280-character limit (hmmm). To send a full message, you might send multiple individual packets with some metadata about which message you’re sending.

Now, back to the idea of routing. Once you have these messages, how does the internet know where to send them? Here, we introduce the idea of packet forwarding. To understand this, suppose we want to drive down from Berkeley to LA without using maps. To do so, we might stop along the way – in Berkeley, someone might tell you directions to get to an Oakland bakery. There, someone might tell you directions to get to a San Jose taco shop. We can keep repeating this until we get to our destination in LA.

Routing functions similarly. A packet sent has a final destination on it, and at each step on the way, different routers and switches send that packet to its next destination. To do so, each router maintains its own table called a forwarding table to figure out where next to send the packet.

As a side note, the method in which this table gets set is complicated, so we will not be going into it. Suffice it to know that this exists, and to dive into it when you really need to!

Check out traceroute to see this in action!

Today’s Network

Let’s go back and figure out the ??? along our network. Armed with the information above, we might imagine some kind of hub-and-spoke architecture, where consumers pay various ISPs (internet service providers, like Comcast or Sonic), which in turn pay one Global ISP to links them all. This is Model 1. For various reasons (political, economic), this is not how the world works. In particular, if being a Global ISP is profitable, then someone else will want to be one too, creating more.

So we turn to Model 2. In this model, we add more Global ISPs that all link to one another. This model is pretty much the same as the previous model in all other ways. However, it’s not quite complete, either; the reality of it is (as always) a bit more complicated.

In reality, it looks a bit more like this:

[user] --> Access ISP --> Regional ISP --> Tier 1 (Global) ISP

, with the arrows indicating the organization they’re paying for access. In some cases, Regional ISPs can pay other Regional ISPs for access, especially in cases of city-wide ISPs paying provincial ISPs, etc.

So the final picture looks like this:

[your phone] ---> Access ISP --> Regional ISP --> Tier 1 ISP ---> [server]
[friend's phone] ---> Access ISP --> Regional ISP --> Tier 1 ISP ---^

Protocol

Great, now we all have an idea of how the internet is all interconnected. Let’s hash out a bit more what actually gets sent, and how it gets sent. Broadly speaking, there are 5 levels that we can examine this at:

  • Application: What is sent by a particular application (HTTP, SMTP, DNS, etc.) This is the format of a message and the message itself. (In our analogy from earlier, think of this as the writing on a page and how to decipher it.)
  • Transport: How a particular message is sent, and how it will arrive (TCP & UDP). This provides connection-oriented service to applications. (In our analogy from earlier, think of this as the paper itself and also things like delivery guarantees.)
  • Network: Responsible for delivery by figuring out where things should be sent and moving them there. Today, this is the IP Protocol, which assigns everyone a unique address. (In our analogy, think of this as logistics companies.)
  • Link: Responsible for providing service to move packet for the Network layer. Today, this is the Ethernet protocol, WiFi protocol, etc.
  • Physical: Responsible for implementing the Link layer in hardware.

You might notice that everything from the Network layer down isn’t particularly useful for most software engineering. So, we’ll be overing the top two layers today!

TCP vs UDP

TCP and UDP are the two transport layer protocols in wide use today. TCP is a connection-oriented transport mechanism, whereas UDP is a best-effort transport mechanism. In practice, this means that

  • TCP guarantees delivery, whereas UDP does not.
  • Both implement error checking.
  • TCP guarantees ordering, whereas UDP does not.
  • TCP implements things like throttling, whereas UDP does not.
  • TCP packets are much bigger than UDP packets (overhead)
  • The TCP system is much more complicated than UDP.

Both are used widely in industry. TCP is used in things where you need guarantee of delivery, like

  • the Web
  • SSH
  • Email protocols

UDP is used when you don’t care about delivery, but instead care about raw speed:

  • Games
  • Media streaming
  • Probably (hopefully) Zoom

If you’re curious about TCP guarantees, read up on the TCP handshake!

The Application Layer

The Application Layer is where backend engineers spend most of their time, where things like HTTP are implemented. Let’s take a look at how HTTP works!

First, we can tell netcat to listen on port 8080 to see what the browser sends over. By default, netcat listens with TCP:

nc -l 8080

Then, in your browser, go to localhost:8080. You should see something like this printed out in netcat’s output:

$ nc -l 8080
GET / HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Cache-Control: max-age=0
DNT: 1
Upgrade-Insecure-Requests: 1
...

Great! Everything printed out is the output of the HTTP Application Layer Protocol. In your browser, you should see nothing (and your browser should still be loading). Let’s paste some output so that our browser can load something. Put the following in a file called response. This is a simple reply that follows the HTTP Protocol.

HTTP/1.1 200
Server: netcat
Content-Type: text/html; charset=UTF-8

Hello World

Then, in your terminal, run a netcat command that will send this response when someone connects:

cat response | nc -l 8080

Now when you connect, you should be able to see Hello World printed in your browser!

An Aside: Wireshark

If you’re curious what’s sent on the wire at the TCP level, check out Wireshark! You’ll be able to analyze all packets sent between netcat and your browser. In class, I’ll demo this on my computer.

Netcat to Netcat

Open two terminals on your computer. In one, run

nc -l 8080

and in the other, run

nc localhost 8080

The two will now be connected! Anything you send on one should show up in the other.

Python to Netcat

Now, we explore using Python to do socket programming. Put the following in server.py:

import socket
# https://docs.python.org/3/howto/sockets.html

# create an INET, STREAMing socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# bind the socket to a public host, and a well-known port
serversocket.bind(("0.0.0.0", 8080))
# become a server socket
serversocket.listen(5)

while True:
    # accept connections from outside
    (clientsocket, address) = serversocket.accept()
    # now do something with the clientsocket

    data = clientsocket.recv(2048)
    while len(data):
        print(data.decode())
        data = clientsocket.recv(2048)
        clientsocket.send(data.upper())

This will listen on port 8080, and once it makes a connection, it will respond with the capitalized version of the message it receives.

To demonstrate this, connect with

nc localhost 8080

and try sending messages.

Our Own Protocol

This concept is pretty powerful! By listening at the socket level, we can create our own protocol. For example, we can make a protocol that keeps track of refrigerator contents, supporting 3 commands:

  • ADD <INGREDIENT> <NUMBER>
  • DEL <INGREDIENT> <NUMBER>
  • GET <INGREDIENT>
import socket
# https://docs.python.org/3/howto/sockets.html

# create an INET, STREAMing socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# bind the socket to a public host, and a well-known port
serversocket.bind(("0.0.0.0", 40002))
# become a server socket
serversocket.listen(5)

while True:
    # accept connections from outside
    (clientsocket, address) = serversocket.accept()
    # now do something with the clientsocket

    data = clientsocket.recv(2048).decode()
    ingredients = {}
    while len(data):
        print(data)
        data_parts = data.split()

        instruction = data_parts[0]
        ingredient = data_parts[1]
        if len(data_parts) > 2:
            number = int(data_parts[2])

        if instruction == "ADD":
            ingredients[ingredient] = ingredients.get(ingredient, 0) + number
        elif instruction == "DEL":
            ingredients[ingredient] = ingredients.get(ingredient, 0) - number
        elif instruction == "GET":
            clientsocket.send((str(ingredients.get(ingredient, 0)) + "\n").encode())

        print(ingredients)
        data = clientsocket.recv(2048).decode()