How the Web Works

Goals

• High level: what happens when you visit a URL in browser

• Explain what IP and DNS are

• Describe the different parts of a URL

• Describe the request / response cycle

• Compare GET vs POST requests

What happens when...

When I type http://site.com/some/page.html into a browser, what really happens?

This is a common interview question for software engineers.

How the Web Works

The internet is complicated. Really, really complicated. Fortunately, to be a software developer, you only need to know a bit. For people who want to work in "development operations," or as a system administrator, it's typical to have to learn more about the details here.

Hostnames

We often talk to servers by "hostname" — site.com or computer-a.site.com. That's just a nickname for the server, though — and the same server can have many hostnames.

IP Addresses

On networks, computers have an "IP Address" — a unique address to find that computer on the network. IP addresses look like 123.77.32.121, four numbers (0–255) connected by dots. There are a lot of advanced edges here that make this more complicated, but most of these details aren't important for software engineers:

• there is another whole way to specify networks, "IPv6," that use a different numbering scheme.

• some computers can have multiple IP addresses they can be reached by

• under some circumstances, multiple computers can share an IP address and have this be handled by a special kind of router. If you're interested in system administration details, you can learn about this by reading about "Network Address Translation."

127.0.0.1

127.0.0.1 is special—it's "this computer that you're on." In addition to their IP address on the network, all computers can reach themselves at this address. The name **localhost ** always maps to 127.0.0.1.

Protocols

"Protocols" are the conventions and ways of one thing talking to another.

http—Hypertext Transfer Protocol (standard web) (How browsers and servers communicate)

https—HTTP Secure (How browsers and servers communicate with encryption)

ftp-File transfer protocol (An older protocol for sending files over internet)

There are many others, but these are the common ones. In this lecture, we'll focus only on HTTP

Hostname

DNS (domain name service) turns this into an IP address

So site.com might resolve to 123.45.67.89

Port

• Every server has 65,535 unique "ports" you can talk to

• Services tend to have a default port

• For HTTP, is port 80

• For HTTPS, is port 443

• You don't have to specify in URL unless you want a different port

  • To do: http://site.com:12345/some/page.html

Resource

• This always talks to some "web server" program on the server

  • For some servers, may have them read an actual file on disk: /some/page.html

  • For many servers, "dynamically generates" a page

Query String

• This provides "extra information" — search terms, info from forms, etc

  • The server is provided this info; might use to change page

  • Sometimes, JavaScript will use this information in addition/instead

• Multiple arguments are separated by &: ?x=i&y=2

  • Argument can be given several times: ?x=i&x=2

So

http://site.com/some/page.html?x=7

means

• Turn "site.com" into 123.45.67.89

• Connect to 123.45.67.89

• On port 80 (default)

• Using HTTP protocol

• Ask for /some/page.html

• Pass along query string: x=1

Request and Response

When you point your browser to a webpage on a server, your browser makes a request to that server. This is almost always a GET request, and it contains the exact URL you want.

The server then responds with the exact HTML for that page:

It's often the case, though, that the web server itself will have to do some work to get the page you want, often interacting with other things, such as database servers.

And then it can give back the response you want:

What's in a Request

• Method (ex: GET)

• HTTP protocol version (almost always 1.1)

• Resource URL you want

• Headers

  • Hostname you're asking about

  • Date your browser thinks it is

  • Language your browser wants information in

  • Any cookies that server has sent

  • And more!

What's in a Response

• HTTP protocol version (almost always 1.1)

• Response Status Code (200,404, etc)

• Headers

  • Content Type (typically text/html for web pages)

  • Date/time the server thinks it is

  • Any cookie server wants to set

  • Any caching information

  • And more!

Watch a Request/Response

Response Codes

200-OK

301-What you requested is elsewhere

404—Not Found

500—Server has an internal problem

Sample HTML

demo/demo.html

CSS

demo/demo.html

Connects to site.com on port 80 and requests:

Response:

Image

demo/demo.html

Connects to tinyurl.com() on port 80 and requests:

JavaScript

Connects to site.com on port 80 and requests:

Hey, That's a Lot of Work!

• Yes, it is

• Requesting one webpage often involves many requests!

• Browsers issue these requests asynchronously

  • They'll assemble the final result as requests come back

• You can view this in browser console --> Network

Curl (OSX)

OSX systems come with a utility, curl, which will make an HTTP request on the command line.

Hey...

Everything is a string!