W3cubDocs

/D

std.net.curl

Networking client functionality as provided by libcurl. The libcurl library must be installed on the system in order to use this module.



Category Functions
High level download upload get post put del options trace connect byLine byChunk byLineAsync byChunkAsync
Low level HTTP FTP SMTP
Note
You may need to link to the curl library, e.g. by adding "libs": ["curl"] to your dub.json file if you are using DUB.
Windows x86 note: A DMD compatible libcurl static library can be downloaded from the dlang.org download page. Compared to using libcurl directly this module allows simpler client code for common uses, requires no unsafe operations, and integrates better with the rest of the language. Futhermore it provides range access to protocols supported by libcurl both synchronously and asynchronously. A high level and a low level API are available. The high level API is built entirely on top of the low level one. The high level API is for commonly used functionality such as HTTP/FTP get. The byLineAsync and byChunkAsync provides asynchronous ranges that performs the request in another thread while handling a line/chunk in the current thread. The low level API allows for streaming and other advanced features.
Cheat Sheet
Function Name Description
High level
download download("ftp.digitalmars.com/sieve.ds", "/tmp/downloaded-ftp-file") downloads file from URL to file system.
upload upload("/tmp/downloaded-ftp-file", "ftp.digitalmars.com/sieve.ds"); uploads file from file system to URL.
get get("dlang.org") returns a char[] containing the dlang.org web page.
put put("dlang.org", "Hi") returns a char[] containing the dlang.org web page. after a HTTP PUT of "hi"
post post("dlang.org", "Hi") returns a char[] containing the dlang.org web page. after a HTTP POST of "hi"
byLine byLine("dlang.org") returns a range of char[] containing the dlang.org web page.
byChunk byChunk("dlang.org", 10) returns a range of ubyte[10] containing the dlang.org web page.
byLineAsync byLineAsync("dlang.org") returns a range of char[] containing the dlang.org web page asynchronously.
byChunkAsync byChunkAsync("dlang.org", 10) returns a range of ubyte[10] containing the dlang.org web page asynchronously.
Low level
HTTP HTTP struct for advanced usage
FTP FTP struct for advanced usage
SMTP SMTP struct for advanced usage
Example
import std.net.curl, std.stdio;

// Return a char[] containing the content specified by a URL
auto content = get("dlang.org");

// Post data and return a char[] containing the content specified by a URL
auto content = post("mydomain.com/here.cgi", ["name1" : "value1", "name2" : "value2"]);

// Get content of file from ftp server
auto content = get("ftp.digitalmars.com/sieve.ds");

// Post and print out content line by line. The request is done in another thread.
foreach (line; byLineAsync("dlang.org", "Post data"))
    writeln(line);

// Get using a line range and proxy settings
auto client = HTTP();
client.proxy = "1.2.3.4";
foreach (line; byLine("dlang.org", client))
    writeln(line);
For more control than the high level functions provide, use the low level API:
Example
import std.net.curl, std.stdio;

// GET with custom data receivers
auto http = HTTP("dlang.org");
http.onReceiveHeader =
    (in char[] key, in char[] value) { writeln(key, ": ", value); };
http.onReceive = (ubyte[] data) { /+ drop +/ return data.length; };
http.perform();
First, an instance of the reference-counted HTTP struct is created. Then the custom delegates are set. These will be called whenever the HTTP instance receives a header and a data buffer, respectively. In this simple example, the headers are written to stdout and the data is ignored. If the request should be stopped before it has finished then return something less than data.length from the onReceive callback. See onReceiveHeader/onReceive for more information. Finally the HTTP request is effected by calling perform(), which is synchronous.
Source
std/net/curl.d
License:
Boost License 1.0.
Authors:
Jonas Drewsen. Some of the SMTP code contributed by Jimmy Cao.
Credits
The functionally is based on libcurl. LibCurl is licensed under an MIT/X derivative license.
struct AutoProtocol
Example
import std.net.curl;
// Two requests below will do the same.
string content;

// Explicit connection provided
content = get!HTTP("dlang.org");

// Guess connection type by looking at the URL
content = get!AutoProtocol("ftp://foo.com/file");
// and since AutoProtocol is default this is the same as
content = get("ftp://foo.com/file");
// and will end up detecting FTP from the url and be the same as
content = get!FTP("ftp://foo.com/file");
void download(Conn = AutoProtocol)(const(char)[] url, string saveToPath, Conn conn = Conn())

Constraints:
if (isCurlConn!Conn)

HTTP/FTP download to local file system.

Parameters:
const(char)[] url resource to download
string saveToPath path to store the downloaded content on local disk
Conn conn connection to use e.g. FTP or HTTP. The default AutoProtocol will guess connection type and create a new instance for this call only.
Example
import std.net.curl;
download("d-lang.appspot.com/testUrl2", "/tmp/downloaded-http-file");
void upload(Conn = AutoProtocol)(string loadFromPath, const(char)[] url, Conn conn = Conn())

Constraints:
if (isCurlConn!Conn)

Upload file from local files system using the HTTP or FTP protocol.

Parameters:
string loadFromPath path load data from local disk.
const(char)[] url resource to upload to
Conn conn connection to use e.g. FTP or HTTP. The default AutoProtocol will guess connection type and create a new instance for this call only.
Example
import std.net.curl;
upload("/tmp/downloaded-ftp-file", "ftp.digitalmars.com/sieve.ds");
upload("/tmp/downloaded-http-file", "d-lang.appspot.com/testUrl2");
T[] get(Conn = AutoProtocol, T = char)(const(char)[] url, Conn conn = Conn())

Constraints:
if (isCurlConn!Conn && (is(T == char) || is(T == ubyte)))

HTTP/FTP get content.

Parameters:
const(char)[] url resource to get
Conn conn connection to use e.g. FTP or HTTP. The default AutoProtocol will guess connection type and create a new instance for this call only. The template parameter T specifies the type to return. Possible values are char and ubyte to return char[] or ubyte[]. If asking for char, content will be converted from the connection character set (specified in HTTP response headers or FTP connection properties, both ISO-8859-1 by default) to UTF-8.
Example
import std.net.curl;
auto content = get("d-lang.appspot.com/testUrl2");
Returns:
A T[] range containing the content of the resource pointed to by the URL.
Throws:
CurlException on error.
See Also:
HTTP.Method
T[] post(T = char, PostUnit)(const(char)[] url, const(PostUnit)[] postData, HTTP conn = HTTP())
T[] post(T = char)(const(char)[] url, string[string] postDict, HTTP conn = HTTP())

Constraints:
if (is(T == char) || is(T == ubyte))
if (is(T == char) || is(T == ubyte))

HTTP post content.

Parameters:
const(char)[] url resource to post to
string[string] postDict data to send as the body of the request. An associative array of string is accepted and will be encoded using www-form-urlencoding
const(PostUnit)[] postData data to send as the body of the request. An array of an arbitrary type is accepted and will be cast to ubyte[] before sending it.
HTTP conn HTTP connection to use
T The template parameter T specifies the type to return. Possible values are char and ubyte to return char[] or ubyte[]. If asking for char, content will be converted from the connection character set (specified in HTTP response headers or FTP connection properties, both ISO-8859-1 by default) to UTF-8.
Examples:
import std.net.curl;

auto content1 = post("d-lang.appspot.com/testUrl2", ["name1" : "value1", "name2" : "value2"]);
auto content2 = post("d-lang.appspot.com/testUrl2", [1,2,3,4]);
Returns:
A T[] range containing the content of the resource pointed to by the URL.
See Also:
HTTP.Method
T[] put(Conn = AutoProtocol, T = char, PutUnit)(const(char)[] url, const(PutUnit)[] putData, Conn conn = Conn())

Constraints:
if (isCurlConn!Conn && (is(T == char) || is(T == ubyte)))

HTTP/FTP put content.

Parameters:
const(char)[] url resource to put
const(PutUnit)[] putData data to send as the body of the request. An array of an arbitrary type is accepted and will be cast to ubyte[] before sending it.
Conn conn connection to use e.g. FTP or HTTP. The default AutoProtocol will guess connection type and create a new instance for this call only. The template parameter T specifies the type to return. Possible values are char and ubyte to return char[] or ubyte[]. If asking for char, content will be converted from the connection character set (specified in HTTP response headers or FTP connection properties, both ISO-8859-1 by default) to UTF-8.
Example
import std.net.curl;
auto content = put("d-lang.appspot.com/testUrl2",
                     "Putting this data");
Returns:
A T[] range containing the content of the resource pointed to by the URL.
See Also:
HTTP.Method
void del(Conn = AutoProtocol)(const(char)[] url, Conn conn = Conn())

Constraints:
if (isCurlConn!Conn)

HTTP/FTP delete content.

Parameters:
const(char)[] url resource to delete
Conn conn connection to use e.g. FTP or HTTP. The default AutoProtocol will guess connection type and create a new instance for this call only.
Example
import std.net.curl;
del("d-lang.appspot.com/testUrl2");
See Also:
HTTP.Method
T[] options(T = char)(const(char)[] url, HTTP conn = HTTP())

Constraints:
if (is(T == char) || is(T == ubyte))

HTTP options request.

Parameters:
const(char)[] url resource make a option call to
HTTP conn connection to use e.g. FTP or HTTP. The default AutoProtocol will guess connection type and create a new instance for this call only. The template parameter T specifies the type to return. Possible values are char and ubyte to return char[] or ubyte[].
Example
import std.net.curl;
auto http = HTTP();
options("d-lang.appspot.com/testUrl2", http);
writeln("Allow set to " ~ http.responseHeaders["Allow"]);
Returns:
A T[] range containing the options of the resource pointed to by the URL.
See Also:
HTTP.Method
T[] trace(T = char)(const(char)[] url, HTTP conn = HTTP())

Constraints:
if (is(T == char) || is(T == ubyte))

HTTP trace request.

Parameters:
const(char)[] url resource make a trace call to
HTTP conn connection to use e.g. FTP or HTTP. The default AutoProtocol will guess connection type and create a new instance for this call only. The template parameter T specifies the type to return. Possible values are char and ubyte to return char[] or ubyte[].
Example
import std.net.curl;
trace("d-lang.appspot.com/testUrl1");
Returns:
A T[] range containing the trace info of the resource pointed to by the URL.
See Also:
HTTP.Method
T[] connect(T = char)(const(char)[] url, HTTP conn = HTTP())

Constraints:
if (is(T == char) || is(T == ubyte))

HTTP connect request.

Parameters:
const(char)[] url resource make a connect to
HTTP conn HTTP connection to use The template parameter T specifies the type to return. Possible values are char and ubyte to return char[] or ubyte[].
Example
import std.net.curl;
connect("d-lang.appspot.com/testUrl1");
Returns:
A T[] range containing the connect info of the resource pointed to by the URL.
See Also:
HTTP.Method
T[] patch(T = char, PatchUnit)(const(char)[] url, const(PatchUnit)[] patchData, HTTP conn = HTTP())

Constraints:
if (is(T == char) || is(T == ubyte))

HTTP patch content.

Parameters:
const(char)[] url resource to patch
const(PatchUnit)[] patchData data to send as the body of the request. An array of an arbitrary type is accepted and will be cast to ubyte[] before sending it.
HTTP conn HTTP connection to use The template parameter T specifies the type to return. Possible values are char and ubyte to return char[] or ubyte[].
Example
auto http = HTTP();
http.addRequestHeader("Content-Type", "application/json");
auto content = patch("d-lang.appspot.com/testUrl2", `{"title": "Patched Title"}`, http);
Returns:
A T[] range containing the content of the resource pointed to by the URL.
See Also:
HTTP.Method
auto byLine(Conn = AutoProtocol, Terminator = char, Char = char)(const(char)[] url, KeepTerminator keepTerminator = No.keepTerminator, Terminator terminator = '\x0a', Conn conn = Conn())

Constraints:
if (isCurlConn!Conn && isSomeChar!Char && isSomeChar!Terminator)

HTTP/FTP fetch content as a range of lines.

A range of lines is returned when the request is complete. If the method or other request properties is to be customized then set the conn parameter with a HTTP/FTP instance that has these properties set.

Example
import std.net.curl, std.stdio;
foreach (line; byLine("dlang.org"))
    writeln(line);
Parameters:
const(char)[] url The url to receive content from
KeepTerminator keepTerminator Yes.keepTerminator signals that the line terminator should be returned as part of the lines in the range.
Terminator terminator The character that terminates a line
Conn conn The connection to use e.g. HTTP or FTP.
Returns:
A range of Char[] with the content of the resource pointer to by the URL
auto byChunk(Conn = AutoProtocol)(const(char)[] url, size_t chunkSize = 1024, Conn conn = Conn())

Constraints:
if (isCurlConn!Conn)

HTTP/FTP fetch content as a range of chunks.

A range of chunks is returned when the request is complete. If the method or other request properties is to be customized then set the conn parameter with a HTTP/FTP instance that has these properties set.

Example
import std.net.curl, std.stdio;
foreach (chunk; byChunk("dlang.org", 100))
    writeln(chunk); // chunk is ubyte[100]
Parameters:
const(char)[] url The url to receive content from
size_t chunkSize The size of each chunk
Conn conn The connection to use e.g. HTTP or FTP.
Returns:
A range of ubyte[chunkSize] with the content of the resource pointer to by the URL
auto byLineAsync(Conn = AutoProtocol, Terminator = char, Char = char, PostUnit)(const(char)[] url, const(PostUnit)[] postData, KeepTerminator keepTerminator = No.keepTerminator, Terminator terminator = '\x0a', size_t transmitBuffers = 10, Conn conn = Conn())
auto byLineAsync(Conn = AutoProtocol, Terminator = char, Char = char)(const(char)[] url, KeepTerminator keepTerminator = No.keepTerminator, Terminator terminator = '\x0a', size_t transmitBuffers = 10, Conn conn = Conn())

Constraints:
if (isCurlConn!Conn && isSomeChar!Char && isSomeChar!Terminator)

HTTP/FTP fetch content as a range of lines asynchronously.

A range of lines is returned immediately and the request that fetches the lines is performed in another thread. If the method or other request properties is to be customized then set the conn parameter with a HTTP/FTP instance that has these properties set.

If postData is non-null the method will be set to post for HTTP requests.

The background thread will buffer up to transmitBuffers number of lines before it stops receiving data from network. When the main thread reads the lines from the range it frees up buffers and allows for the background thread to receive more data from the network.

If no data is available and the main thread accesses the range it will block until data becomes available. An exception to this is the wait(Duration) method on the AsyncLineInputRange. This method will wait at maximum for the specified duration and return true if data is available.

Example
import std.net.curl, std.stdio;
// Get some pages in the background
auto range1 = byLineAsync("www.google.com");
auto range2 = byLineAsync("www.wikipedia.org");
foreach (line; byLineAsync("dlang.org"))
    writeln(line);

// Lines already fetched in the background and ready
foreach (line; range1) writeln(line);
foreach (line; range2) writeln(line);

import std.net.curl, std.stdio;
// Get a line in a background thread and wait in
// main thread for 2 seconds for it to arrive.
auto range3 = byLineAsync("dlang.com");
if (range3.wait(dur!"seconds"(2)))
    writeln(range3.front);
else
    writeln("No line received after 2 seconds!");
Parameters:
const(char)[] url The url to receive content from
const(PostUnit)[] postData Data to HTTP Post
KeepTerminator keepTerminator Yes.keepTerminator signals that the line terminator should be returned as part of the lines in the range.
Terminator terminator The character that terminates a line
size_t transmitBuffers The number of lines buffered asynchronously
Conn conn The connection to use e.g. HTTP or FTP.
Returns:
A range of Char[] with the content of the resource pointer to by the URL.
auto byChunkAsync(Conn = AutoProtocol, PostUnit)(const(char)[] url, const(PostUnit)[] postData, size_t chunkSize = 1024, size_t transmitBuffers = 10, Conn conn = Conn())
auto byChunkAsync(Conn = AutoProtocol)(const(char)[] url, size_t chunkSize = 1024, size_t transmitBuffers = 10, Conn conn = Conn())

Constraints:
if (isCurlConn!Conn)
if (isCurlConn!Conn)

HTTP/FTP fetch content as a range of chunks asynchronously.

A range of chunks is returned immediately and the request that fetches the chunks is performed in another thread. If the method or other request properties is to be customized then set the conn parameter with a HTTP/FTP instance that has these properties set.

If postData is non-null the method will be set to post for HTTP requests.

The background thread will buffer up to transmitBuffers number of chunks before is stops receiving data from network. When the main thread reads the chunks from the range it frees up buffers and allows for the background thread to receive more data from the network.

If no data is available and the main thread access the range it will block until data becomes available. An exception to this is the wait(Duration) method on the AsyncChunkInputRange. This method will wait at maximum for the specified duration and return true if data is available.

Example
import std.net.curl, std.stdio;
// Get some pages in the background
auto range1 = byChunkAsync("www.google.com", 100);
auto range2 = byChunkAsync("www.wikipedia.org");
foreach (chunk; byChunkAsync("dlang.org"))
    writeln(chunk); // chunk is ubyte[100]

// Chunks already fetched in the background and ready
foreach (chunk; range1) writeln(chunk);
foreach (chunk; range2) writeln(chunk);

import std.net.curl, std.stdio;
// Get a line in a background thread and wait in
// main thread for 2 seconds for it to arrive.
auto range3 = byChunkAsync("dlang.com", 10);
if (range3.wait(dur!"seconds"(2)))
    writeln(range3.front);
else
    writeln("No chunk received after 2 seconds!");
Parameters:
const(char)[] url The url to receive content from
const(PostUnit)[] postData Data to HTTP Post
size_t chunkSize The size of the chunks
size_t transmitBuffers The number of chunks buffered asynchronously
Conn conn The connection to use e.g. HTTP or FTP.
Returns:
A range of ubyte[chunkSize] with the content of the resource pointer to by the URL.
struct HTTP

HTTP client functionality.

Example
import std.net.curl, std.stdio;

// Get with custom data receivers
auto http = HTTP("dlang.org");
http.onReceiveHeader =
    (in char[] key, in char[] value) { writeln(key ~ ": " ~ value); };
http.onReceive = (ubyte[] data) { /+ drop +/ return data.length; };
http.perform();

// Put with data senders
auto msg = "Hello world";
http.contentLength = msg.length;
http.onSend = (void[] data)
{
    auto m = cast(void[]) msg;
    size_t len = m.length > data.length ? data.length : m.length;
    if (len == 0) return len;
    data[0 .. len] = m[0 .. len];
    msg = msg[len..$];
    return len;
};
http.perform();

// Track progress
http.method = HTTP.Method.get;
http.url = "http://upload.wikimedia.org/wikipedia/commons/"
           "5/53/Wikipedia-logo-en-big.png";
http.onReceive = (ubyte[] data) { return data.length; };
http.onProgress = (size_t dltotal, size_t dlnow,
                   size_t ultotal, size_t ulnow)
{
    writeln("Progress ", dltotal, ", ", dlnow, ", ", ultotal, ", ", ulnow);
    return 0;
};
http.perform();
See Also:
RFC2616
alias AuthMethod = etc.c.curl.CurlAuth

Authentication method equal to etc.c.curl.CurlAuth

alias TimeCond = etc.c.curl.CurlTimeCond

Time condition enumeration as an alias of etc.c.curl.CurlTimeCond

RFC2616 Section 14.25

static HTTP opCall(const(char)[] url)

Constructor taking the url as parameter.

static HTTP opCall()
HTTP dup()
CurlCode perform(ThrowOnError throwOnError = Yes.throwOnError)

Perform a http request.

After the HTTP client has been setup and possibly assigned callbacks the perform() method will start performing the request towards the specified server.

Parameters:
ThrowOnError throwOnError whether to throw an exception or return a CurlCode on error
@property void url(const(char)[] url)

The URL to specify the location of the resource.

@property void caInfo(const(char)[] caFile)

Set the CA certificate bundle file to use for SSL peer verification

alias requestPause = etc.c.curl.CurlReadFunc.pause

Value to return from onSend/onReceive delegates in order to pause a request

alias requestAbort = etc.c.curl.CurlReadFunc.abort

Value to return from onSend delegate in order to abort a request

@property bool isStopped()

True if the instance is stopped. A stopped instance is not usable.

void shutdown()

Stop and invalidate this instance.

@property void verbose(bool on)

Set verbose. This will print request information to stderr.

@property void dataTimeout(Duration d)

Set timeout for activity on connection.

@property void operationTimeout(Duration d)

Set maximum time an operation is allowed to take. This includes dns resolution, connecting, data transfer, etc.

@property void connectTimeout(Duration d)

Set timeout for connecting.

@property void proxy(const(char)[] host)

Proxy

See
proxy
@property void proxyPort(ushort port)

Proxy port

See
proxy_port
alias CurlProxy = etc.c.curl.CurlProxy

Type of proxy

@property void proxyType(CurlProxy type)

Proxy type

See
proxy_type
@property void dnsTimeout(Duration d)

DNS lookup timeout.

@property void netInterface(const(char)[] i)
@property void netInterface(const(ubyte)[4] i)
@property void netInterface(InternetAddress i)

The network interface to use in form of the the IP of the interface.

Example
theprotocol.netInterface = "192.168.1.32";
theprotocol.netInterface = [ 192, 168, 1, 32 ];
See
std.socket.InternetAddress
@property void localPort(ushort port)

Set the local outgoing port to use.

Parameters:
ushort port the first outgoing port number to try and use
@property void localPortRange(ushort range)

Set the local outgoing port range to use. This can be used together with the localPort property.

Parameters:
ushort range if the first port is occupied then try this many port number forwards
@property void tcpNoDelay(bool on)

Set the tcp no-delay socket option on or off.

See
nodelay
void setAuthentication(const(char)[] username, const(char)[] password, const(char)[] domain = "")

Set the user name, password and optionally domain for authentication purposes.

Some protocols may need authentication in some cases. Use this function to provide credentials.

Parameters:
const(char)[] username the username
const(char)[] password the password
const(char)[] domain used for NTLM authentication only and is set to the NTLM domain name
void setProxyAuthentication(const(char)[] username, const(char)[] password)

Set the user name and password for proxy authentication.

Parameters:
const(char)[] username the username
const(char)[] password the password
@property void onSend(size_t delegate(void[]) callback)

The event handler that gets called when data is needed for sending. The length of the void[] specifies the maximum number of bytes that can be sent.

Returns:
The callback returns the number of elements in the buffer that have been filled and are ready to send. The special value .abortRequest can be returned in order to abort the current request. The special value .pauseRequest can be returned in order to pause the current request.
Example
import std.net.curl;
string msg = "Hello world";
auto client = HTTP("dlang.org");
client.onSend = delegate size_t(void[] data)
{
    auto m = cast(void[]) msg;
    size_t length = m.length > data.length ? data.length : m.length;
    if (length == 0) return 0;
    data[0 .. length] = m[0 .. length];
    msg = msg[length..$];
    return length;
};
client.perform();
@property void onReceive(size_t delegate(ubyte[]) callback)

The event handler that receives incoming data. Be sure to copy the incoming ubyte[] since it is not guaranteed to be valid after the callback returns.

Returns:
The callback returns the incoming bytes read. If not the entire array is the request will abort. The special value .pauseRequest can be returned in order to pause the current request.
Example
import std.net.curl, std.stdio;
auto client = HTTP("dlang.org");
client.onReceive = (ubyte[] data)
{
    writeln("Got data", to!(const(char)[])(data));
    return data.length;
};
client.perform();
@property void onProgress(int delegate(size_t dlTotal, size_t dlNow, size_t ulTotal, size_t ulNow) callback)

Register an event handler that gets called to inform of upload/download progress.

Callback parameters
dlTotal total bytes to download
dlNow currently downloaded bytes
ulTotal total bytes to upload
ulNow currently uploaded bytes
Connection type used when the URL should be used to auto detect the protocol. This struct is used as placeholder for the connection parameter when calling the high level API and the connection type (HTTP/FTP) should be guessed by inspecting the URL parameter. The rules for guessing the protocol are: 1, if URL starts with ftp://, ftps:// or ftp. then FTP connection is assumed. 2, HTTP connection otherwise.
Callback returns
Return 0 to signal success, return non-zero to abort transfer.
Example
import std.net.curl, std.stdio;
auto client = HTTP("dlang.org");
client.onProgress = delegate int(size_t dl, size_t dln, size_t ul, size_t ult)
{
    writeln("Progress: downloaded ", dln, " of ", dl);
    writeln("Progress: uploaded ", uln, " of ", ul);
};
client.perform();
void clearRequestHeaders()

Clear all outgoing headers.

void addRequestHeader(const(char)[] name, const(char)[] value)

Add a header e.g. "X-CustomField: Something is fishy".

There is no remove header functionality. Do a clearRequestHeaders and set the needed headers instead.

Example
import std.net.curl;
auto client = HTTP();
client.addRequestHeader("X-Custom-ABC", "This is the custom value");
auto content = get("dlang.org", client);
static @property string defaultUserAgent()

The default "User-Agent" value send with a request. It has the form "Phobos-std.net.curl/PHOBOS_VERSION (libcurl/CURL_VERSION)"

void setUserAgent(const(char)[] userAgent)

Set the value of the user agent request header field.

By default a request has it's "User-Agent" field set to defaultUserAgent even if setUserAgent was never called. Pass an empty string to suppress the "User-Agent" field altogether.

CurlCode getTiming(CurlInfo timing, ref double val)

Get various timings defined in etc.c.curl.CurlInfo. The value is usable only if the return value is equal to etc.c.curl.CurlError.ok.

Parameters:
CurlInfo timing one of the timings defined in etc.c.curl.CurlInfo. The values are: etc.c.curl.CurlInfo.namelookup_time, etc.c.curl.CurlInfo.connect_time, etc.c.curl.CurlInfo.pretransfer_time, etc.c.curl.CurlInfo.starttransfer_time, etc.c.curl.CurlInfo.redirect_time, etc.c.curl.CurlInfo.appconnect_time, etc.c.curl.CurlInfo.total_time.
double val the actual value of the inquired timing.
Returns:
The return code of the operation. The value stored in val should be used only if the return value is etc.c.curl.CurlInfo.ok.
Example
import std.net.curl;
import etc.c.curl : CurlError, CurlInfo;

auto client = HTTP("dlang.org");
client.perform();

double val;
CurlCode code;

code = http.getTiming(CurlInfo.namelookup_time, val);
assert(code == CurlError.ok);
@property string[string] responseHeaders()

The headers read from a successful response.

@property void method(Method m)
@property Method method()

HTTP method used.

@property StatusLine statusLine()

HTTP status line of last response. One call to perform may result in several requests because of redirection.

void setCookie(const(char)[] cookie)

Set the active cookie string e.g. "name1=value1;name2=value2"

void setCookieJar(const(char)[] path)

Set a file path to where a cookie jar should be read/stored.

void flushCookieJar()

Flush cookie jar to disk.

void clearSessionCookies()

Clear session cookies.

void clearAllCookies()

Clear all cookies.

void setTimeCondition(HTTP.TimeCond cond, SysTime timestamp)

Set time condition on the request.

Parameters:
HTTP.TimeCond cond CurlTimeCond.{none,ifmodsince,ifunmodsince,lastmod}
SysTime timestamp Timestamp for the condition RFC2616 Section 14.25
@property void postData(const(void)[] data)

Specifying data to post when not using the onSend callback.

The data is NOT copied by the library. Content-Type will default to application/octet-stream. Data is not converted or encoded by this method.

Example
import std.net.curl, std.stdio;
auto http = HTTP("http://www.mydomain.com");
http.onReceive = (ubyte[] data) { writeln(to!(const(char)[])(data)); return data.length; };
http.postData = [1,2,3,4,5];
http.perform();
@property void postData(const(char)[] data)

Specifying data to post when not using the onSend callback.

The data is NOT copied by the library. Content-Type will default to text/plain. Data is not converted or encoded by this method.

Example
import std.net.curl, std.stdio;
auto http = HTTP("http://www.mydomain.com");
http.onReceive = (ubyte[] data) { writeln(to!(const(char)[])(data)); return data.length; };
http.postData = "The quick....";
http.perform();
void setPostData(const(void)[] data, string contentType)

Specify data to post when not using the onSend callback, with user-specified Content-Type.

Parameters:
const(void)[] data Data to post.
string contentType MIME type of the data, for example, "text/plain" or "application/octet-stream". See also: Internet media type on Wikipedia. 
import std.net.curl;
auto http = HTTP("http://onlineform.example.com");
auto data = "app=login&username=bob&password=s00perS3kret";
http.setPostData(data, "application/x-www-form-urlencoded");
http.onReceive = (ubyte[] data) { return data.length; };
http.perform();
@property void onReceiveHeader(void delegate(in char[] key, in char[] value) callback)

Set the event handler that receives incoming headers.

The callback will receive a header field key, value as parameter. The const(char)[] arrays are not valid after the delegate has returned.

Example
import std.net.curl, std.stdio;
auto http = HTTP("dlang.org");
http.onReceive = (ubyte[] data) { writeln(to!(const(char)[])(data)); return data.length; };
http.onReceiveHeader = (in char[] key, in char[] value) { writeln(key, " = ", value); };
http.perform();
@property void onReceiveStatusLine(void delegate(StatusLine) callback)

Callback for each received StatusLine.

Notice that several callbacks can be done for each call to perform() due to redirections.

See Also:
StatusLine
@property void contentLength(ulong len)

The content length in bytes when using request that has content e.g. POST/PUT and not using chunked transfer. Is set as the "Content-Length" header. Set to ulong.max to reset to chunked transfer.

@property void authenticationMethod(AuthMethod authMethod)

Authentication method as specified in AuthMethod.

@property void maxRedirects(uint maxRedirs)

Set max allowed redirections using the location header. uint.max for infinite.

enum Method: int

The standard HTTP methods : RFC2616 Section 5.1.1

head
get
post
put
del
options
trace
connect
patch
struct StatusLine

HTTP status line ie. the first line returned in an HTTP response.

If authentication or redirections are done then the status will be for the last response received.

ushort majorVersion

Major HTTP version ie. 1 in HTTP/1.0.

ushort minorVersion

Minor HTTP version ie. 0 in HTTP/1.0.

ushort code

HTTP status line code e.g. 200.

string reason

HTTP status line reason string.

@safe void reset()

Reset this status line

const string toString()
struct FTP

FTP client functionality.

See Also:
RFC959
static FTP opCall(const(char)[] url)

FTP access to the specified url.

static FTP opCall()
FTP dup()
CurlCode perform(ThrowOnError throwOnError = Yes.throwOnError)

Performs the ftp request as it has been configured.

After a FTP client has been setup and possibly assigned callbacks the perform() method will start performing the actual communication with the server.

Parameters:
ThrowOnError throwOnError whether to throw an exception or return a CurlCode on error
@property void url(const(char)[] url)

The URL to specify the location of the resource.

alias requestPause = etc.c.curl.CurlReadFunc.pause

Value to return from onSend/onReceive delegates in order to pause a request

alias requestAbort = etc.c.curl.CurlReadFunc.abort

Value to return from onSend delegate in order to abort a request

@property bool isStopped()

True if the instance is stopped. A stopped instance is not usable.

void shutdown()

Stop and invalidate this instance.

@property void verbose(bool on)

Set verbose. This will print request information to stderr.

@property void dataTimeout(Duration d)

Set timeout for activity on connection.

@property void operationTimeout(Duration d)

Set maximum time an operation is allowed to take. This includes dns resolution, connecting, data transfer, etc.

@property void connectTimeout(Duration d)

Set timeout for connecting.

@property void proxy(const(char)[] host)

Proxy

See
proxy
@property void proxyPort(ushort port)

Proxy port

See
proxy_port
alias CurlProxy = etc.c.curl.CurlProxy

Type of proxy

@property void proxyType(CurlProxy type)

Proxy type

See
proxy_type
@property void dnsTimeout(Duration d)

DNS lookup timeout.

@property void netInterface(const(char)[] i)
@property void netInterface(const(ubyte)[4] i)
@property void netInterface(InternetAddress i)

The network interface to use in form of the the IP of the interface.

Example
theprotocol.netInterface = "192.168.1.32";
theprotocol.netInterface = [ 192, 168, 1, 32 ];
See
std.socket.InternetAddress
@property void localPort(ushort port)

Set the local outgoing port to use.

Parameters:
ushort port the first outgoing port number to try and use
@property void localPortRange(ushort range)

Set the local outgoing port range to use. This can be used together with the localPort property.

Parameters:
ushort range if the first port is occupied then try this many port number forwards
@property void tcpNoDelay(bool on)

Set the tcp no-delay socket option on or off.

See
nodelay
void setAuthentication(const(char)[] username, const(char)[] password, const(char)[] domain = "")

Set the user name, password and optionally domain for authentication purposes.

Some protocols may need authentication in some cases. Use this function to provide credentials.

Parameters:
const(char)[] username the username
const(char)[] password the password
const(char)[] domain used for NTLM authentication only and is set to the NTLM domain name
void setProxyAuthentication(const(char)[] username, const(char)[] password)

Set the user name and password for proxy authentication.

Parameters:
const(char)[] username the username
const(char)[] password the password
@property void onSend(size_t delegate(void[]) callback)

The event handler that gets called when data is needed for sending. The length of the void[] specifies the maximum number of bytes that can be sent.

Returns:
The callback returns the number of elements in the buffer that have been filled and are ready to send. The special value .abortRequest can be returned in order to abort the current request. The special value .pauseRequest can be returned in order to pause the current request.
@property void onReceive(size_t delegate(ubyte[]) callback)

The event handler that receives incoming data. Be sure to copy the incoming ubyte[] since it is not guaranteed to be valid after the callback returns.

Returns:
The callback returns the incoming bytes read. If not the entire array is the request will abort. The special value .pauseRequest can be returned in order to pause the current request.
@property void onProgress(int delegate(size_t dlTotal, size_t dlNow, size_t ulTotal, size_t ulNow) callback)

The event handler that gets called to inform of upload/download progress.

Callback parameters
dlTotal total bytes to download
dlNow currently downloaded bytes
ulTotal total bytes to upload
ulNow currently uploaded bytes
Connection type used when the URL should be used to auto detect the protocol. This struct is used as placeholder for the connection parameter when calling the high level API and the connection type (HTTP/FTP) should be guessed by inspecting the URL parameter. The rules for guessing the protocol are: 1, if URL starts with ftp://, ftps:// or ftp. then FTP connection is assumed. 2, HTTP connection otherwise.
Callback returns
Return 0 from the callback to signal success, return non-zero to abort transfer.
void clearCommands()

Clear all commands send to ftp server.

void addCommand(const(char)[] command)

Add a command to send to ftp server.

There is no remove command functionality. Do a clearCommands and set the needed commands instead.

Example
import std.net.curl;
auto client = FTP();
client.addCommand("RNFR my_file.txt");
client.addCommand("RNTO my_renamed_file.txt");
upload("my_file.txt", "ftp.digitalmars.com", client);
@property void encoding(string name)
@property string encoding()

Connection encoding. Defaults to ISO-8859-1.

@property void contentLength(ulong len)

The content length in bytes of the ftp data.

CurlCode getTiming(CurlInfo timing, ref double val)

Get various timings defined in etc.c.curl.CurlInfo. The value is usable only if the return value is equal to etc.c.curl.CurlError.ok.

Parameters:
CurlInfo timing one of the timings defined in etc.c.curl.CurlInfo. The values are: etc.c.curl.CurlInfo.namelookup_time, etc.c.curl.CurlInfo.connect_time, etc.c.curl.CurlInfo.pretransfer_time, etc.c.curl.CurlInfo.starttransfer_time, etc.c.curl.CurlInfo.redirect_time, etc.c.curl.CurlInfo.appconnect_time, etc.c.curl.CurlInfo.total_time.
double val the actual value of the inquired timing.
Returns:
The return code of the operation. The value stored in val should be used only if the return value is etc.c.curl.CurlInfo.ok.
Example
import std.net.curl;
import etc.c.curl : CurlError, CurlInfo;

auto client = FTP();
client.addCommand("RNFR my_file.txt");
client.addCommand("RNTO my_renamed_file.txt");
upload("my_file.txt", "ftp.digitalmars.com", client);

double val;
CurlCode code;

code = http.getTiming(CurlInfo.namelookup_time, val);
assert(code == CurlError.ok);
struct SMTP

Basic SMTP protocol support.

Example
import std.net.curl;

// Send an email with SMTPS
auto smtp = SMTP("smtps://smtp.gmail.com");
smtp.setAuthentication("[email protected]", "password");
smtp.mailTo = ["<[email protected]>"];
smtp.mailFrom = "<[email protected]>";
smtp.message = "Example Message";
smtp.perform();
See Also:
RFC2821
static SMTP opCall(const(char)[] url)

Sets to the URL of the SMTP server.

static SMTP opCall()
CurlCode perform(ThrowOnError throwOnError = Yes.throwOnError)

Performs the request as configured.

Parameters:
ThrowOnError throwOnError whether to throw an exception or return a CurlCode on error
@property void url(const(char)[] url)

The URL to specify the location of the resource.

alias requestPause = etc.c.curl.CurlReadFunc.pause

Value to return from onSend/onReceive delegates in order to pause a request

alias requestAbort = etc.c.curl.CurlReadFunc.abort

Value to return from onSend delegate in order to abort a request

@property bool isStopped()

True if the instance is stopped. A stopped instance is not usable.

void shutdown()

Stop and invalidate this instance.

@property void verbose(bool on)

Set verbose. This will print request information to stderr.

@property void dataTimeout(Duration d)

Set timeout for activity on connection.

@property void operationTimeout(Duration d)

Set maximum time an operation is allowed to take. This includes dns resolution, connecting, data transfer, etc.

@property void connectTimeout(Duration d)

Set timeout for connecting.

@property void proxy(const(char)[] host)

Proxy

See
proxy
@property void proxyPort(ushort port)

Proxy port

See
proxy_port
alias CurlProxy = etc.c.curl.CurlProxy

Type of proxy

@property void proxyType(CurlProxy type)

Proxy type

See
proxy_type
@property void dnsTimeout(Duration d)

DNS lookup timeout.

@property void netInterface(const(char)[] i)
@property void netInterface(const(ubyte)[4] i)
@property void netInterface(InternetAddress i)

The network interface to use in form of the the IP of the interface.

Example
theprotocol.netInterface = "192.168.1.32";
theprotocol.netInterface = [ 192, 168, 1, 32 ];
See
std.socket.InternetAddress
@property void localPort(ushort port)

Set the local outgoing port to use.

Parameters:
ushort port the first outgoing port number to try and use
@property void localPortRange(ushort range)

Set the local outgoing port range to use. This can be used together with the localPort property.

Parameters:
ushort range if the first port is occupied then try this many port number forwards
@property void tcpNoDelay(bool on)

Set the tcp no-delay socket option on or off.

See
nodelay
void setAuthentication(const(char)[] username, const(char)[] password, const(char)[] domain = "")

Set the user name, password and optionally domain for authentication purposes.

Some protocols may need authentication in some cases. Use this function to provide credentials.

Parameters:
const(char)[] username the username
const(char)[] password the password
const(char)[] domain used for NTLM authentication only and is set to the NTLM domain name
void setProxyAuthentication(const(char)[] username, const(char)[] password)

Set the user name and password for proxy authentication.

Parameters:
const(char)[] username the username
const(char)[] password the password
@property void onSend(size_t delegate(void[]) callback)

The event handler that gets called when data is needed for sending. The length of the void[] specifies the maximum number of bytes that can be sent.

Returns:
The callback returns the number of elements in the buffer that have been filled and are ready to send. The special value .abortRequest can be returned in order to abort the current request. The special value .pauseRequest can be returned in order to pause the current request.
@property void onReceive(size_t delegate(ubyte[]) callback)

The event handler that receives incoming data. Be sure to copy the incoming ubyte[] since it is not guaranteed to be valid after the callback returns.

Returns:
The callback returns the incoming bytes read. If not the entire array is the request will abort. The special value .pauseRequest can be returned in order to pause the current request.
@property void onProgress(int delegate(size_t dlTotal, size_t dlNow, size_t ulTotal, size_t ulNow) callback)

The event handler that gets called to inform of upload/download progress.

Callback parameters
dlTotal total bytes to download
dlNow currently downloaded bytes
ulTotal total bytes to upload
ulNow currently uploaded bytes
Connection type used when the URL should be used to auto detect the protocol. This struct is used as placeholder for the connection parameter when calling the high level API and the connection type (HTTP/FTP) should be guessed by inspecting the URL parameter. The rules for guessing the protocol are: 1, if URL starts with ftp://, ftps:// or ftp. then FTP connection is assumed. 2, HTTP connection otherwise.
Callback returns
Return 0 from the callback to signal success, return non-zero to abort transfer.
@property void mailFrom()(const(char)[] sender)

Setter for the sender's email address.

void mailTo()(const(char)[][] recipients...)

Setter for the recipient email addresses.

@property void message(string msg)

Sets the message body text.

class CurlException: object.Exception

Exception thrown on errors in std.net.curl functions.

pure nothrow @safe this(string msg, string file = __FILE__, size_t line = __LINE__, Throwable next = null)
Parameters:
string msg The message for the exception.
string file The file where the exception occurred.
size_t line The line number where the exception occurred.
Throwable next The previous exception in the chain of exceptions, if any.
class CurlTimeoutException: std.net.curl.CurlException

Exception thrown on timeout errors in std.net.curl functions.

pure nothrow @safe this(string msg, string file = __FILE__, size_t line = __LINE__, Throwable next = null)
Parameters:
string msg The message for the exception.
string file The file where the exception occurred.
size_t line The line number where the exception occurred.
Throwable next The previous exception in the chain of exceptions, if any.
class HTTPStatusException: std.net.curl.CurlException

Exception thrown on HTTP request failures, e.g. 404 Not Found.

pure nothrow @safe this(int status, string msg, string file = __FILE__, size_t line = __LINE__, Throwable next = null)
Parameters:
int status The HTTP status code.
string msg The message for the exception.
string file The file where the exception occurred.
size_t line The line number where the exception occurred.
Throwable next The previous exception in the chain of exceptions, if any.
immutable int status

The HTTP status code

alias CurlCode = int

Equal to etc.c.curl.CURLcode

alias ThrowOnError = std.typecons.Flag!"throwOnError".Flag

Flag to specify whether or not an exception is thrown on error.

struct Curl

Wrapper to provide a better interface to libcurl than using the plain C API. It is recommended to use the HTTP/FTP etc. structs instead unless raw access to libcurl is needed.

Warning
This struct uses interior pointers for callbacks. Only allocate it on the stack if you never move or copy it. This also means passing by reference when passing Curl to other functions. Otherwise always allocate on the heap.
void initialize()

Initialize the instance by creating a working curl handle.

const @property bool stopped()
Curl dup()

Duplicate this handle.

The new handle will have all options set as the one it was duplicated from. An exception to this is that all options that cannot be shared across threads are reset thereby making it safe to use the duplicate in a new thread.

void shutdown()

Stop and invalidate this curl instance.

Warning
Do not call this from inside a callback handler e.g. onReceive.
void pause(bool sendingPaused, bool receivingPaused)

Pausing and continuing transfers.

void set(CurlOption option, const(char)[] value)

Set a string curl option.

Parameters:
CurlOption option A etc.c.curl.CurlOption as found in the curl documentation
const(char)[] value The string
void set(CurlOption option, long value)

Set a long curl option.

Parameters:
CurlOption option A etc.c.curl.CurlOption as found in the curl documentation
long value The long
void set(CurlOption option, void* value)

Set a void* curl option.

Parameters:
CurlOption option A etc.c.curl.CurlOption as found in the curl documentation
void* value The pointer
void clear(CurlOption option)

Clear a pointer option.

Parameters:
CurlOption option A etc.c.curl.CurlOption as found in the curl documentation
void clearIfSupported(CurlOption option)

Clear a pointer option. Does not raise an exception if the underlying libcurl does not support the option. Use sparingly.

Parameters:
CurlOption option A etc.c.curl.CurlOption as found in the curl documentation
CurlCode perform(ThrowOnError throwOnError = Yes.throwOnError)

perform the curl request by doing the HTTP,FTP etc. as it has been setup beforehand.

Parameters:
ThrowOnError throwOnError whether to throw an exception or return a CurlCode on error
CurlCode getTiming(CurlInfo timing, ref double val)

Get the various timings like name lookup time, total time, connect time etc. The timed category is passed through the timing parameter while the timing value is stored at val. The value is usable only if res is equal to etc.c.curl.CurlError.ok.

@property void onReceive(size_t delegate(InData) callback)

The event handler that receives incoming data.

Parameters:
size_t delegate(InData) callback the callback that receives the ubyte[] data. Be sure to copy the incoming data and not store a slice.
Returns:
The callback returns the incoming bytes read. If not the entire array is the request will abort. The special value HTTP.pauseRequest can be returned in order to pause the current request.
Example
import std.net.curl, std.stdio;
Curl curl;
curl.initialize();
curl.set(CurlOption.url, "http://dlang.org");
curl.onReceive = (ubyte[] data) { writeln("Got data", to!(const(char)[])(data)); return data.length;};
curl.perform();
@property void onReceiveHeader(void delegate(in char[]) callback)

The event handler that receives incoming headers for protocols that uses headers.

Parameters:
void delegate(in char[]) callback the callback that receives the header string. Make sure the callback copies the incoming params if it needs to store it because they are references into the backend and may very likely change.
Example
import std.net.curl, std.stdio;
Curl curl;
curl.initialize();
curl.set(CurlOption.url, "http://dlang.org");
curl.onReceiveHeader = (in char[] header) { writeln(header); };
curl.perform();
@property void onSend(size_t delegate(OutData) callback)

The event handler that gets called when data is needed for sending.

Parameters:
size_t delegate(OutData) callback the callback that has a void[] buffer to be filled
Returns:
The callback returns the number of elements in the buffer that have been filled and are ready to send. The special value Curl.abortRequest can be returned in order to abort the current request. The special value Curl.pauseRequest can be returned in order to pause the current request.
Example
import std.net.curl;
Curl curl;
curl.initialize();
curl.set(CurlOption.url, "http://dlang.org");

string msg = "Hello world";
curl.onSend = (void[] data)
{
    auto m = cast(void[]) msg;
    size_t length = m.length > data.length ? data.length : m.length;
    if (length == 0) return 0;
    data[0 .. length] = m[0 .. length];
    msg = msg[length..$];
    return length;
};
curl.perform();
@property void onSeek(CurlSeek delegate(long, CurlSeekPos) callback)

The event handler that gets called when the curl backend needs to seek the data to be sent.

Parameters:
CurlSeek delegate(long, CurlSeekPos) callback the callback that receives a seek offset and a seek position etc.c.curl.CurlSeekPos
Returns:
The callback returns the success state of the seeking etc.c.curl.CurlSeek
Example
import std.net.curl;
Curl curl;
curl.initialize();
curl.set(CurlOption.url, "http://dlang.org");
curl.onSeek = (long p, CurlSeekPos sp)
{
    return CurlSeek.cantseek;
};
curl.perform();
@property void onSocketOption(int delegate(curl_socket_t, CurlSockType) callback)

The event handler that gets called when the net socket has been created but a connect() call has not yet been done. This makes it possible to set misc. socket options.

Parameters:
int delegate(curl_socket_t, CurlSockType) callback the callback that receives the socket and socket type etc.c.curl.CurlSockType
Returns:
Return 0 from the callback to signal success, return 1 to signal error and make curl close the socket
Example
import std.net.curl;
Curl curl;
curl.initialize();
curl.set(CurlOption.url, "http://dlang.org");
curl.onSocketOption = delegate int(curl_socket_t s, CurlSockType t) { /+ do stuff +/ };
curl.perform();
@property void onProgress(int delegate(size_t dlTotal, size_t dlNow, size_t ulTotal, size_t ulNow) callback)

The event handler that gets called to inform of upload/download progress.

Parameters:
int delegate(size_t dlTotal, size_t dlNow, size_t ulTotal, size_t ulNow) callback the callback that receives the (total bytes to download, currently downloaded bytes, total bytes to upload, currently uploaded bytes).
Returns:
Return 0 from the callback to signal success, return non-zero to abort transfer
Example
import std.net.curl;
Curl curl;
curl.initialize();
curl.set(CurlOption.url, "http://dlang.org");
curl.onProgress = delegate int(size_t dltotal, size_t dlnow, size_t ultotal, size_t uln)
{
    writeln("Progress: downloaded bytes ", dlnow, " of ", dltotal);
    writeln("Progress: uploaded bytes ", ulnow, " of ", ultotal);
    curl.perform();
};

© 1999–2017 The D Language Foundation
Licensed under the Boost License 1.0.
https://dlang.org/phobos/std_net_curl.html