document.util Module

Misc functions.

wpull.document.util.detect_response_encoding(response, is_html=False, peek=131072)[source]

Return the likely encoding of the response document.

Parameters:
  • response (Response) – An instance of http.Response.
  • is_html (bool) – See util.detect_encoding().
  • peek (int) – The maximum number of bytes of the document to be analyzed.
Returns:

The codec name.
Return type:

str, None
wpull.document.util.get_heading_encoding(response)[source]

Return the document encoding from a HTTP header.

Parameters:response (Response) – An instance of http.Response.
Returns:The codec name.
Return type:str, None
wpull.document.util.is_gzip(data)[source]

Return whether the data is likely to be gzip.