processor.ftp Module

FTP

class wpull.processor.ftp.FTPProcessor(ftp_client: wpull.protocol.ftp.client.Client, fetch_params)[source]

Bases: wpull.processor.base.BaseProcessor

FTP processor.

Parameters:
  • ftp_client – The FTP client.
  • fetch_params (WebProcessorFetchParams) – Parameters for fetching.
close()[source]

Close the FTP client.

fetch_params

The fetch parameters.

ftp_client

The ftp client.

listing_cache

Listing cache.

Returns:A cache mapping from URL to list of ftp.ls.listing.FileEntry.
process(item_session: wpull.pipeline.session.ItemSession)[source]
wpull.processor.ftp.FTPProcessorFetchParams

FTPProcessorFetchParams

Parameters:
  • remove_listing (bool) – Remove .listing files after fetching.
  • glob (bool) – Enable URL globbing.
  • preserve_permissions (bool) – Preserve file permissions.
  • follow_symlinks (bool) – Follow symlinks.

alias of FTPProcessorFetchParamsType

class wpull.processor.ftp.FTPProcessorSession(processor: wpull.processor.ftp.FTPProcessor, item_session: wpull.pipeline.session.ItemSession)[source]

Bases: wpull.processor.base.BaseProcessorSession

Fetches FTP files or directory listings.

close()[source]
process()[source]

Process.

Coroutine.

exception wpull.processor.ftp.HookPreResponseBreak[source]

Bases: wpull.errors.ProtocolError

Hook pre-response break.

wpull.processor.ftp.append_slash_to_path_url(url_info: wpull.url.URLInfo) → str[source]

Return URL string with the path suffixed with a slash.

wpull.processor.ftp.to_dir_path_url(url_info: wpull.url.URLInfo) → str[source]

Return URL string with the path replaced with directory only.