Is printing the raw http request from the built in http/https libraries not supported?

[1mike12]

When using the built in http libraries as a client, I'm shocked that there is no way to see the raw request as it will be rendered when sent out. Unless I missed it, I don't think it's possible? The conversations that discuss this topic on stack overflow for instance, only give hacky ways where you setup a man in the middle to listen in. https://stackoverflow.com/questions/53214169/node-js-request-print-entire-http-request-raw-of-a-post . Internally, node is generating this request, surely there is a way to expose this back up to the caller?

There are sometimes subtle bugs that arise due to how the final output http request is formatted. For instance, sometimes header order can impact an end server behaves.

here's an example of how it's handled in other frameworks/languages

https://pkg.go.dev/net/http#Request.Write

req, err := http.NewRequest("GET", "https://httpbin.org/get", nil)
if err != nil {
    panic(err)
}
req.Header.Add("User-Agent", "Go-HttpClient/1.1")

var buf bytes.Buffer
err = req.Write(&buf)
if err != nil {
    panic(err)
}

rawBytes := buf.Bytes()
fmt.Println(string(rawBytes))

https://docs.python-requests.org/en/latest/api/#requests.PreparedRequest

import requests
req = requests.Request('GET', 'https://httpbin.org/get')
r = req.prepare()
print(r)

[1mike12]

After poking around, I see that the requests are streamed out which I was not thinking of. I was only thinking of the simple cases of making a GET or POST with lots of headers, not anything with a huge body, which I imagined could quickly be toString()'d . I've never looked at node source but there must be a way to possibly even pass a writeable stream into a debug function in order to view the raw request that way?

node/lib/_http_outgoing.js

Line 904 in 82ac335

const ret = write_(this, chunk, encoding, callback, false);

[sunnypatell]

You're correct that Node.js does not provide a built-in way to retrieve the fully formatted HTTP request before it is sent. This limitation arises because requests are streamed rather than constructed as a full string in memory.

However, you can achieve similar behavior using a writable stream as an interceptor. By using http.ClientRequest's event listeners, you can capture chunks as they are written. Here's a simple approach:

const http = require('http');
const { Writable } = require('stream');

class CaptureStream extends Writable {
  constructor() {
    super();
    this.chunks = [];
  }
  _write(chunk, encoding, callback) {
    this.chunks.push(chunk);
    callback();
  }
  getRawRequest() {
    return Buffer.concat(this.chunks).toString();
  }
}

const options = {
  hostname: 'httpbin.org',
  path: '/get',
  method: 'GET',
  headers: { 'User-Agent': 'Node.js' },
};

const req = http.request(options, (res) => {
  res.on('data', () => {});
  res.on('end', () => {
    console.log('Response received');
  });
});

const captureStream = new CaptureStream();
req.on('socket', (socket) => {
  socket.on('connect', () => {
    console.log('Raw Request:\n', captureStream.getRawRequest());
  });
});

req.end();

This isn't a perfect one-to-one equivalent of Go's Request.Write() or Python’s PreparedRequest, but it provides a way to capture outgoing request data.

Alternative Approach: Hooking Into _http_outgoing.js
If you're comfortable with modifying internals, you could explore patching OutgoingMessage.prototype.write in _http_outgoing.js, which would allow capturing the raw HTTP request before it's sent over the wire. However, this isn't an ideal approach as it modifies internal behavior.

Feature Proposal?
It would be useful if Node exposed an API similar to Go's Request.Write(), but given Node’s streaming architecture, it would likely require buffering the request explicitly before sending. A middleware-style hook for debugging outgoing requests might be worth proposing.