An in-depth look at the Node HTTP module, and how to use it to scale up!

The HTTP module is a core module of Node.js, it is fair to say it's one of the biggest responsible for Node's initial rise in popularity.

The Hypertext Transfer Protocol (HTTP) is an application-level protocol for distributed, collaborative, hypermedia information systems. It is a generic, stateless, protocol which can be used for many tasks beyond its use for hypertext, such as name servers and distributed object management systems, through extension of its request methods, error codes and headers 

. A feature of HTTP is the typing and negotiation of data representation, allowing systems to be built independently of the data being transferred.

HTTP has been in use by the World-Wide Web global information initiative since 1990. This specification defines the protocol referred to as "HTTP/1.1", and is an update to RFC2616.

HTTP is the veins of the internet, this website, any site you explore, you use the HTTP protocol to request it from a server, and the server uses the same HTTP protocol to send you back the data you requested.

Your first HTTP server using the HTTP module

Let's import the module and create a basic HTTP server with it

// We import the http module
const http = require('http');

// we create a new server instance
const Server = http.createServer((request, response) => {
  // This function will be triggered on each request
});

// We set our server to listen for requests in port 3000
Server.listen(3000);

We just created an HTTP Server object, some of its methods are:

While your server is still active and listening to new connections, the 

 function you passed will be active in the event loop. This is the reason why our preview code snippet doesn't automatically exit after executing the last line of code. A node application will stay alive until there aren't more lines of code to execute or the event loop is still populated.

 then the event loop gets cleared, and the application will end.

For a complete list of HTTP server class properties and methods, check out the 

Handling HTTP requests with the node http module

 function to handle HTTP requests and respond to them.

: shorthand for request, is an object from the class  

 that includes all the request information. Some interesting properties are:

 class, so each request object is, indeed, a stream.

: shorthand for "response", is an object from the class ServerResponse, which contains a collection of methods to send back an HTTP response. Some of the methods are:

Each time we write response data with .write(), the chunk of data we passed gets immediately flushed to the kernel buffer. If we try to .write() after .end() has been called, we will get the following error:

Handling POST requests with the http module

In an HTTP post request, we usually get a body as part of the request. How do we access it using the Node HTTP module?

Remember the request object is an instance of the 

 class, in post request we can access the body as a stream of data like this:

, to launch the HTTP request to our application, and you would end up with something like this:

While you are in Postman, be sure that you are firing POST request. You will also need to set the following configuration in order to set the 

To handle HTTPS requests, Node.js has the core module 

, but first, we need some SSL certificates, for the purpose of this example let's generate some self-signed certificates

In your command line (use GitBash if you are on windows) lets run

openssl genrsa -out key.pem
openssl req -new -key key.pem -out csr.pem
openssl x509 -req -days 9999 -in csr.pem -signkey key.pem -out cert.pem
rm csr.pem

Now let's use the HTTPS module to create an HTTP server

The main difference is we now have to read the key files and load them into an options object, that we pass to 

Sending a request using the node HTTP module

Sometimes we would like to do an HTTP request in order to gather data from a third-party HTTP server. We can achieve this by using the Node HTTP module 

In the following example, we will be calling the postman-echo API which returns whatever we send them.

const http = require("http")

// The body object we will be sending
let body = JSON.stringify({
  title: "Sending data!"
})

http
  .request(
  	// Configuration object with request details
    {
      hostname: "postman-echo.com",
      path: "/post",
  	  method: "POST",
      headers: {
        "Content-Type": "application/json",
        "Content-Length": Buffer.byteLength(body)
  	  }
    },
    res => {
      let response = ""

      res.on("data", chunk => {
        response += chunk
      })
      res.on("end", () => {
        console.log(response)
      })
    }
  )
  // here we attach our body to be sent
  .end(body)

But I would suggest instead of using the core HTTP module for sending requests, you use something more sophisticated and user friendly as 

The pros of using something like Axios, is promise abstraction, easier to manage errors on requests, and support for really valuable plugins, like Axios retry.

Use Express don't reinvent the wheel with 

In a lot of situations, we can use a framework like Express to create a server instead of doing directly to the HTTP module.

Install express using npm in your command line

const express = require('express')
const app = express()
const port = 3000

app.get('/', (req, res) => {
  res.send('Hello World!')
})

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`)
})

Express will give you a more elegant way to handle all your API routes, handle session data, and will provide you some plugins for authentication, Express is gonna make your life way easier! 

We've been through a lot, within this short post, but with these few examples, you should have a good grasp on the core functionalities of the HTTP Node core module.

It's ideal to understand the inner functionalities of the HTTP module, but for more complex tasks is recommended to use an abstraction library, such as express in the case you are building servers, or Axios in the case you are creating HTTP requests.

Check out the documentation of the modules we used in this post:

If you have any questions - or want feedback on your post - come join 

<h1 data-node-type="titlePlaceholder" class="titlePlaceholder">Deep dive into the Node HTTP module</h1><div data-node-type="summary" class="summary-block">An in-depth look at the Node HTTP module, and how to use it to scale up!</div><p>The HTTP module is a core module of Node.js, it is fair to say it's one of the biggest responsible for Node's initial rise in popularity.</p><div data-panel-type="info" data-panel-position="inline" class="panel panel-type-info panel-position-inline"><div class="panel-container lolwut"><h2><a name="what-you-will-learn" id="what-you-will-learn">What You Will Learn</a></h2><ul><li><p>To handle requests using Node and the HTTP module ✅</p></li><li><p>To handle post requests, and parse a body using the HTTP module ✅</p></li><li><p>To create an HTTPS server ✅</p></li><li><p>How to send HTTP requests ✅</p></li></ul></div></div><h1><a name="requirements" id="requirements">Requirements</a></h1><ul><li><p>Some experience with javascript.</p></li><li><p>An Installation of <a href="https://nodejs.org/" rel="noopener noreferrer nofollow">Node.js</a>.</p></li><li><p>Some of you who are using windows should install <a href="https://git-scm.com/downloads" rel="noopener noreferrer nofollow">GitBash</a>.</p></li><li><p><a href="https://www.postman.com/downloads/" rel="noopener noreferrer nofollow">Postman</a> will come in handy to do some HTTP requests.</p></li></ul><h1><a name="overview-of-the-http-protocol" id="overview-of-the-http-protocol">Overview of the HTTP protocol</a></h1><blockquote data-block-type="information" class="information annotated"><p>The Hypertext Transfer Protocol (HTTP) is an application-level protocol for distributed, collaborative, hypermedia information systems. It is a generic, stateless, protocol which can be used for many tasks beyond its use for hypertext, such as name servers and distributed object management systems, through extension of its request methods, error codes and headers&nbsp;<a href="https://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/draft-lafon-rfc2616bis-03.html#RFC2324" rel="noopener noreferrer nofollow">[RFC2324]</a>. A feature of HTTP is the typing and negotiation of data representation, allowing systems to be built independently of the data being transferred.</p><p>HTTP has been in use by the World-Wide Web global information initiative since 1990. This specification defines the protocol referred to as "HTTP/1.1", and is an update to RFC2616.<br><br><a href="https://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/draft-lafon-rfc2616bis-02.html" rel="noopener noreferrer nofollow">W3.org HTTP reference</a></p></blockquote><p>HTTP is the veins of the internet, this website, any site you explore, you use the HTTP protocol to request it from a server, and the server uses the same HTTP protocol to send you back the data you requested.</p><h1><a name="your-first-http-server-using-the-http-module" id="your-first-http-server-using-the-http-module">Your first HTTP server using the HTTP module</a></h1><p>Let's import the module and create a basic HTTP server with it</p><textarea data-iscodemirror="true" data-mode="javascript" data-uniqueid="cm-ZR2Wl" id="cm-ZR2Wl">// We import the http module
const http = require('http');

// we create a new server instance
const Server = http.createServer((request, response) =&gt; {
  // This function will be triggered on each request
});

// We set our server to listen for requests in port 3000
Server.listen(3000);</textarea><p>We just created an HTTP Server object, some of its methods are:</p><ul><li><p><code>Server.listen()</code>  Binds the server to listen to a specific port, this port binding is subject to permission restrictions, Unix TCP/IP ports bellow 1024 are privileged ports, so if you try to listen to port 80 (HTTP default port) your user running the node app might need special permissions, such as <code>sudo</code>.</p></li><li><p><code>Server.close()</code> stops the server from listening to more requests, unbinds the server from the port.</p></li></ul><div data-panel-type="success" data-panel-position="inline" class="panel panel-type-success panel-position-inline"><div class="panel-container lolwut"><p>While your server is still active and listening to new connections, the <code>callback</code> function you passed will be active in the event loop. This is the reason why our preview code snippet doesn't automatically exit after executing the last line of code. A node application will stay alive until there aren't more lines of code to execute or the event loop is still populated.<br><br>But if you call <code>Server.close()</code> then the event loop gets cleared, and the application will end.</p></div></div><p>For a complete list of HTTP server class properties and methods, check out the <a href="https://nodejs.org/api/http.html#http_class_http_server" rel="noopener noreferrer nofollow">official docs</a>. </p><h1><a name="handling-http-requests-with-the-node-http-module" id="handling-http-requests-with-the-node-http-module">Handling HTTP requests with the node http module</a></h1><p>In the example below, let's use the <code>callback</code> function to handle HTTP requests and respond to them.</p><textarea data-iscodemirror="true" data-mode="javascript" data-uniqueid="cm-uztA6" id="cm-uztA6">const http = require('http');

const Server = http.createServer((req, res) =&gt; {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.write('Aronis wishes u happy holidays! \n');
  res.write(`URL from request ${req.url} \n`);
  res.write(`Address from request ${req.connection.remoteAddress} \n`);
  res.write(`Headers from request ${JSON.stringify(req.headers)} \n`);
  res.end();
});

Server.listen(3000);</textarea><p><code>req</code>: shorthand for request, is an object from the class &nbsp;<a href="https://www.w3schools.com/nodejs/obj_http_incomingmessage.asp" rel="noopener noreferrer nofollow">IncomingMessage</a>&nbsp;that includes all the request information. Some interesting properties are:</p><ul><li><p>.connection contains all the connection information pertinent to the request such as .remoteAddress, that contains the network address that originated or forwarded the request.</p></li><li><p>.headers includes the headers attached to the request. If you visited the application at localhost:300 with your browser, one of the headers you would get is "user-agent" which includes information about the browser.</p></li></ul><p>It's also good to remember that the <a href="https://www.w3schools.com/nodejs/obj_http_incomingmessage.asp" rel="noopener noreferrer nofollow">IncomingMessage</a> extends the <a href="https://nodejs.org/api/stream.html#stream_class_stream_readable" rel="noopener noreferrer nofollow">&lt;stream.Readable&gt;</a> class, so each request object is, indeed, a stream.</p><p><code>res</code>: shorthand for "response", is an object from the class ServerResponse, which contains a collection of methods to send back an HTTP response. Some of the methods are:</p><ul><li><p><code>.writeHead()</code> should be called first when initiating a response, allows us to set up the HTTP response code and headers we are sending back.</p></li><li><p>.<code>write(chunk[, encoding][, callback])</code> allows us to send a chunk of data as part of our response. In this case, we declared in the header the content-type as 'text/plain', so each time we write() the data we pass gets attached as a string of text.</p><p><code>chunk</code> could be either a buffer or a string, in the case, it is a string the second parameter which is optional, allows us to set the encoding of the data being sent, the default value is <code>'utf8'</code>.</p><p>The third parameter is a callback function to be executed when the chunk of data has been sent.</p><p>The first call to .write() will send the headers established in .writeHeader() or send the default ones.</p></li><li><p><code>.end()</code> signalizes the response as complete, MUST be called once per response.</p></li></ul><p>Each time we write response data with .write(), the chunk of data we passed gets immediately flushed to the kernel buffer. If we try to .write() after .end() has been called, we will get the following error:</p><div data-panel-type="error" data-panel-position="inline" class="panel panel-type-error panel-position-inline"><div class="panel-container lolwut"><p>Error [ER<em>STREAM</em>WRIT<em>AFTER</em>END]: write after end</p></div></div><h1><a name="handling-post-requests-with-the-http-module" id="handling-post-requests-with-the-http-module">Handling POST requests with the http module</a></h1><p>In an HTTP post request, we usually get a body as part of the request. How do we access it using the Node HTTP module?<br>Remember the request object is an instance of the <a href="https://www.w3schools.com/nodejs/obj_http_incomingmessage.asp" rel="noopener noreferrer nofollow">IncomingMessage</a> class which extends the <a href="https://nodejs.org/api/stream.html#stream_class_stream_readable" rel="noopener noreferrer nofollow">&lt;stream.Readable&gt;</a> class, in post request we can access the body as a stream of data like this:</p><textarea data-iscodemirror="true" data-mode="javascript" data-uniqueid="cm-TZxJN" id="cm-TZxJN">const http = require('http');


const Server = http.createServer( (req, res) =&gt; {
  // If the request is a POST 
  // we try to parse the body
  if(req.method === 'POST'){
    let body = '';

    // We listen to the even data
	// In the request stream for the body
    req.on('data', chunk =&gt; {
      body += chunk.toString();
    });

	// We listen for the end of the stream
    req.on('end', () =&gt; {
      
      // We check the headers
	  // if we are getting a json object
      // We parse it from the string we gen
      if(
        req.headers["content-type"] === 'application/json'
      ) body = JSON.parse(body);
      
      console.log(body);
	  // We must always end the response
      res.end('ok');
    });
  }
});

Server.listen(3000);</textarea><p><br>You could use an application like <a href="https://www.postman.com/" rel="noopener noreferrer nofollow">Postman</a>, to launch the HTTP request to our application, and you would end up with something like this:</p><figure class="photo figure-position-inline"><img src="https://s3.amazonaws.com/assets.fullstack.io/n/20201111025814891_Screenshot_1.png" attachmentid="b9f7d6ae-09dc-406a-86ff-603d4484c434" contenteditable="false" data-width="903" data-height="646"><div></div></figure><p>While you are in Postman, be sure that you are firing POST request. You will also need to set the following configuration in order to set the <code>'content-type'</code> headers as<code> 'application-json'</code></p><figure class="photo figure-position-inline"><img src="https://s3.amazonaws.com/assets.fullstack.io/n/20201111030234503_Screenshot_2.png" attachmentid="0f65bfba-733d-4c62-b859-675290dc4d04" contenteditable="false" data-width="809" data-height="313"><div></div></figure><p></p><h1><a name="creating-an-https-server" id="creating-an-https-server">Creating an HTTPS server</a></h1><p>To handle HTTPS requests, Node.js has the core module <code>https</code>, but first, we need some SSL certificates, for the purpose of this example let's generate some self-signed certificates</p><p>In your command line (use GitBash if you are on windows) lets run</p><textarea data-iscodemirror="true" data-mode="text/x-sh" data-uniqueid="cm-p8InY" id="cm-p8InY">openssl genrsa -out key.pem
openssl req -new -key key.pem -out csr.pem
openssl x509 -req -days 9999 -in csr.pem -signkey key.pem -out cert.pem
rm csr.pem</textarea><p>Now let's use the HTTPS module to create an HTTP server</p><textarea data-iscodemirror="true" data-mode="javascript" data-uniqueid="cm-QAxIz" id="cm-QAxIz">const https = require('https');
const fs = require('fs');

const options = {
  key: fs.readFileSync('key.pem'),
  cert: fs.readFileSync('cert.pem')
};

https.createServer(options, function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.write('Aronis wishes u happy holidays! \n');
  res.write(`URL from request ${req.url} \n`);
  res.write(`Address from request ${req.connection.remoteAddress} \n`);
  res.write(`Headers from request ${JSON.stringify(req.headers)} \n`);
  res.end();
}).listen(3000);</textarea><p>The main difference is we now have to read the key files and load them into an options object, that we pass to <code>.createServer()</code> to create our new shiny HTTPS server.</p><h1><a name="sending-a-request-using-the-node-http-module" id="sending-a-request-using-the-node-http-module">Sending a request using the node HTTP module</a></h1><p>Sometimes we would like to do an HTTP request in order to gather data from a third-party HTTP server. We can achieve this by using the Node HTTP module <code>.request()</code>function.</p><p>In the following example, we will be calling the postman-echo API which returns whatever we send them.</p><textarea data-iscodemirror="true" data-mode="javascript" data-uniqueid="cm-V3lV3" id="cm-V3lV3">const http = require("http")

// The body object we will be sending
let body = JSON.stringify({
  title: "Sending data!"
})

http
  .request(
  	// Configuration object with request details
    {
      hostname: "postman-echo.com",
      path: "/post",
  	  method: "POST",
      headers: {
        "Content-Type": "application/json",
        "Content-Length": Buffer.byteLength(body)
  	  }
    },
    res =&gt; {
      let response = ""

      res.on("data", chunk =&gt; {
        response += chunk
      })
      res.on("end", () =&gt; {
        console.log(response)
      })
    }
  )
  // here we attach our body to be sent
  .end(body)</textarea><p>But I would suggest instead of using the core HTTP module for sending requests, you use something more sophisticated and user friendly as <a href="https://www.newline.co/@AoX04/axios-js-and-nodejs-a-match-made-in-heaven-for-api-consumption--bf0618b1" rel="noopener noreferrer nofollow">Axios</a>.<br>The pros of using something like Axios, is promise abstraction, easier to manage errors on requests, and support for really valuable plugins, like Axios retry.</p><h1><a name="use-express-dont-reinvent-the-wheel-with-httpcreateserver" id="use-express-dont-reinvent-the-wheel-with-httpcreateserver">Use Express don't reinvent the wheel with <code>http.createServer()</code></a></h1><p>In a lot of situations, we can use a framework like Express to create a server instead of doing directly to the HTTP module.<br><br>Install express using npm in your command line</p><textarea data-iscodemirror="true" data-mode="text/x-sh" data-uniqueid="cm-9Iwu5" id="cm-9Iwu5">npm install express --save</textarea><textarea data-iscodemirror="true" data-mode="javascript" data-uniqueid="cm-xOKIC" id="cm-xOKIC">const express = require('express')
const app = express()
const port = 3000

app.get('/', (req, res) =&gt; {
  res.send('Hello World!')
})

app.listen(port, () =&gt; {
  console.log(`Example app listening at http://localhost:${port}`)
})</textarea><p><br>Express will give you a more elegant way to handle all your API routes, handle session data, and will provide you some plugins for authentication, Express is gonna make your life way easier! <br></p><h2><a name="conclusion" id="conclusion">Conclusion</a></h2><p>We've been through a lot, within this short post, but with these few examples, you should have a good grasp on the core functionalities of the HTTP Node core module.<br>It's ideal to understand the inner functionalities of the HTTP module, but for more complex tasks is recommended to use an abstraction library, such as express in the case you are building servers, or Axios in the case you are creating HTTP requests.</p><p>Have fun and keep coding!<br></p><h2><a name="more-resources" id="more-resources">More Resources</a></h2><p>Check out the documentation of the modules we used in this post:</p><ul><li><p><a href="https://github.com/axios/axios" rel="noopener noreferrer nofollow">Axios</a></p></li><li><p><a href="https://expressjs.com/" rel="noopener noreferrer nofollow">Express</a></p></li></ul><h2><a name="getting-help-and-feedback" id="getting-help-and-feedback">Getting Help (and Feedback)</a></h2><p>If you have any questions - or want feedback on your post - come join <a href="https://newline.co/discord" rel="noopener noreferrer nofollow">our community Discord</a>. See you there!</p><p><br><br><br></p><p></p><p></p><p><br></p>

Learn

The newline Guide to Building Your First GraphQL Server with Node and TypeScript

Teach

Amelia Wattenberger

Author of Fullstack D3

Community

Free Tools

Deep dive into the Node HTTP module

Tags

Responses (0)

Masterclasses

Tutorials

Fullstack React with TypeScript