How Can I Use Deflated/gzipped Content With An Xhr Onprogress Function?

May 30, 2024 Post a Comment

I've seen a bunch of similar questions to this get asked before, but I haven't found one that describes my current problem exactly, so here goes: I have a page which loads a large

Solution 1:

A slightly more elegant variation on your solution would be to set a header like 'x-decompressed-content-length' or whatever in your HTTP response with the full decompressed value of the content in bytes and read it off the xhr object in your onProgress handler.

Your code might look something like:

request.onProgress = function (e) {
  var contentLength;
  if (e.lengthComputable) {
    contentLength = e.total;
  } else {
    contentLength = parseInt(e.target.getResponseHeader('x-decompressed-content-length'), 10);
  }
  progressIndicator.update(e.loaded / contentLength);
};

Solution 2:

I wasn't able to solve the issue of using onProgress on the compressed content itself, but I came up with this semi-simple workaround. In a nutshell: send a HEAD request to the server at the same time as a GET request, and render the progress bar once there's enough information to do so.

functionloader(onDone, onProgress, url, data)
{
    // onDone = event handler to run on successful download// onProgress = event handler to run during a download// url = url to load// data = extra parameters to be sent with the AJAX requestvar content_length = null;

    self.meta_xhr = $.ajax({
        url: url,
        data: data,
        dataType: 'json',
        type: 'HEAD',
        success: function(data, status, jqXHR)
        {
            content_length = jqXHR.getResponseHeader("X-Content-Length");
        }
    });

    self.xhr = $.ajax({
        url: url,
        data: data,
        success: onDone,
        dataType: 'json',
        progress: function(jqXHR, evt)
        {
            var pct = 0;
            if (evt.lengthComputable)
            {
                pct = 100 * evt.position / evt.total;
            }
            elseif (self.content_length != null)
            {
                pct = 100 * evt.position / self.content_length;
            }

            onProgress(pct);
        }
    });
}

And then to use it:

loader(function(response)
{
    console.log("Content loaded! do stuff now.");
},
function(pct)
{
    console.log("The content is " + pct + "% loaded.");
},
'<url here>', {});

On the server side, set the X-Content-Length header on both the GET and the HEAD requests (which should represent the uncompressed content length), and abort sending the content on the HEAD request.

In PHP, setting the header looks like:

header("X-Content-Length: ".strlen($payload));

And then abort sending the content if it's a HEAD request:

if ($_SERVER['REQUEST_METHOD'] == "HEAD")
{
    exit;
}

Here's what it looks like in action:

screenshot

The reason the HEAD takes so long in the below screenshot is because the server still has to parse the file to know how long it is, but that's something I can definitely improve on, and it's definitely an improvement from where it was.

Solution 3:

Don't get stuck just because there isn't a native solution; a hack of one line can solve your problem without messing with Apache configuration (that in some hostings is prohibited or very restricted):

PHP to the rescue:

var size = <?phpecho filesize('file.json') ?>;

That's it, you probably already know the rest, but just as a reference here it is:

<script>var progressBar = document.getElementById("p"),
    client = newXMLHttpRequest(),
    size = <?phpecho filesize('file.json') ?>;

progressBar.max = size;

client.open("GET", "file.json")

functionloadHandler () {
  var loaded = client.responseText.length;
  progressBar.value = loaded;
}

client.onprogress = loadHandler;

client.onloadend = function(pe) {
  loadHandler();
  console.log("Success, loaded: " + client.responseText.length + " of " + size)
}
client.send()
</script>

Live example:

Another SO user thinks I am lying about the validity of this solution so here it is live: http://nyudvik.com/zip/, it is gzip-ed and the real file weights 8 MB

Solution 4:

Try changing your server encoding to gzip.

Your request header shows three potential encodings (gzip,deflate,sdch), so the server can pick any one of those three. By the response header, we can see that your server is choosing to respond with deflate.

Gzip is an encoding format that includes a deflate payload in addition to additional headers and footer (which includes the original uncompressed length) and a different checksum algorithm:

Gzip at Wikipedia

Deflate has some issues. Due to legacy issues dealing with improper decoding algorithms, client implementations of deflate have to run through silly checks just to figure out which implementation they're dealing with, and unfortunately, they often still get it wrong:

Why use deflate instead of gzip for text files served by Apache?

In the case of your question, the browser probably sees a deflate file coming down the pipe and just throws up its arms and says, "When I don't even know exactly how I'll end up decoding this thing, how can you expect me to worry about getting the progress right, human?"

If you switch your server configuration so the response is gzipped (i.e., gzip shows up as the content-encoding), I'm hopeful your script works as you'd hoped/expected it would.

Solution 5:

We have created a library that estimates the progress and always sets lengthComputable to true.

Chrome 64 still has this issue (see Bug)

It is a javascript shim that you can include in your page which fixes this issue and you can use the standard new XMLHTTPRequest() normally.

The javascript library can be found here:

https://github.com/AirConsole/xmlhttprequest-length-computable

JavaScript Guide