std.bitmanip

Bit-level manipulation facilities.

Category	Functions
Bit constructs	BitArray bitfields bitsSet
Endianness conversion	bigEndianToNative littleEndianToNative nativeToBigEndian nativeToLittleEndian swapEndian
Integral ranges	append peek read write
Floating-Point manipulation	DoubleRep FloatRep
Tagging	taggedClassRef taggedPointer

License:

Boost License 1.0.

Authors:

Walter Bright, Andrei Alexandrescu, Jonathan M Davis, Alex Rønne Petersen, Damian Ziemba, Amaury SECHET

Source

std/bitmanip.d

template bitfields(T...)

Allows creating bit fields inside structs and classes.

Example

struct A
{
    int a;
    mixin(bitfields!(
        uint, "x",    2,
        int,  "y",    3,
        uint, "z",    2,
        bool, "flag", 1));
}
A obj;
obj.x = 2;
obj.z = obj.x;

The example above creates a bitfield pack of eight bits, which fit in one ubyte. The bitfields are allocated starting from the least significant bit, i.e. x occupies the two least significant bits of the bitfields storage. The sum of all bit lengths in one bitfield instantiation must be exactly 8, 16, 32, or 64. If padding is needed, just allocate one bitfield with an empty name.

Example

struct A
{
    mixin(bitfields!(
        bool, "flag1",    1,
        bool, "flag2",    1,
        uint, "",         6));
}

The type of a bit field can be any integral type or enumerated type. The most efficient type to store in bitfields is bool, followed by unsigned types, followed by signed types.

enum auto taggedPointer(T : T*, string name, Ts...)

This string mixin generator allows one to create tagged pointers inside structs and classes.

A tagged pointer uses the bits known to be zero in a normal pointer or class reference to store extra information. For example, a pointer to an integer must be 4-byte aligned, so there are 2 bits that are always known to be zero. One can store a 2-bit integer there.

The example above creates a tagged pointer in the struct A. The pointer is of type uint* as specified by the first argument, and is named x, as specified by the second argument.

Following arguments works the same way as bitfield's. The bitfield must fit into the bits known to be zero because of the pointer alignment.

Examples:

struct A
{
    int a;
    mixin(taggedPointer!(
        uint*, "x",
        bool, "b1", 1,
        bool, "b2", 1));
}
A obj;
obj.x = new uint;
obj.b1 = true;
obj.b2 = false;

template taggedClassRef(T, string name, Ts...) if (is(T == class))

This string mixin generator allows one to create tagged class reference inside structs and classes.

A tagged class reference uses the bits known to be zero in a normal class reference to store extra information. For example, a pointer to an integer must be 4-byte aligned, so there are 2 bits that are always known to be zero. One can store a 2-bit integer there.

The example above creates a tagged reference to an Object in the struct A. This expects the same parameters as taggedPointer, except the first argument which must be a class type instead of a pointer type.

Examples:

struct A
{
    int a;
    mixin(taggedClassRef!(
        Object, "o",
        uint, "i", 2));
}
A obj;
obj.o = new Object();
obj.i = 3;

struct FloatRep

Allows manipulating the fraction, exponent, and sign parts of a float separately. The definition is:

struct FloatRep
{
    union
    {
        float value;
        mixin(bitfields!(
                  uint,  "fraction", 23,
                  ubyte, "exponent",  8,
                  bool,  "sign",      1));
    }
    enum uint bias = 127, fractionBits = 23, exponentBits = 8, signBits = 1;
}

struct DoubleRep

Allows manipulating the fraction, exponent, and sign parts of a double separately. The definition is:

struct DoubleRep
{
    union
    {
        double value;
        mixin(bitfields!(
                  ulong,   "fraction", 52,
                  ushort,  "exponent", 11,
                  bool,    "sign",      1));
    }
    enum uint bias = 1023, signBits = 1, fractionBits = 52, exponentBits = 11;
}

struct BitArray

An array of bits.

const pure nothrow @nogc @property @safe size_t dim()

Gets the amount of native words backing this BitArray.

const pure nothrow @nogc @property @safe size_t length()

Gets the amount of bits in the BitArray.

pure nothrow @property @system size_t length(size_t newlen)

Sets the amount of bits in the BitArray. Warning: increasing length may overwrite bits in final word up to the next word boundary. i.e. D dynamic array extension semantics are not followed.

const pure nothrow @nogc bool opIndex(size_t i)

Gets the i'th bit in the BitArray.

pure nothrow @nogc bool opIndexAssign(bool b, size_t i)

Sets the i'th bit in the BitArray.

const pure nothrow @property BitArray dup()

Duplicates the BitArray and its contents.

int opApply(scope int delegate(ref bool) dg)
const int opApply(scope int delegate(bool) dg)
int opApply(scope int delegate(size_t, ref bool) dg)
const int opApply(scope int delegate(size_t, bool) dg)

Support for foreach loops for BitArray.

pure nothrow @nogc @property BitArray reverse()

Reverses the bits of the BitArray.

pure nothrow @nogc @property BitArray sort()

Sorts the BitArray's elements.

const pure nothrow @nogc bool opEquals(ref const BitArray a2)

Support for operators == and != for BitArray.

const pure nothrow @nogc int opCmp(BitArray a2)

Supports comparison operators for BitArray.

const pure nothrow @nogc size_t toHash()

Support for hashing for BitArray.

pure nothrow @system this(bool[] ba)

Set this BitArray to the contents of ba.

pure nothrow this(void[] v, size_t numbits)

Map the BitArray onto v, with numbits being the number of bits in the array. Does not copy the data. v.length must be a multiple of size_t.sizeof. If there are unmapped bits in the final mapped word then these will be set to 0.

This is the inverse of opCast.

pure nothrow @nogc void[] opCast(T : void[])()

Convert to void[].

pure nothrow @nogc size_t[] opCast(T : size_t[])()

Convert to size_t[].

const pure nothrow BitArray opCom()

Support for unary operator ~ for BitArray.

const pure nothrow BitArray opBinary(string op)(const BitArray e2)

Constraints:
if (op == "-" || op == "&" || op == "|" || op == "^")

Support for binary bitwise operators for BitArray.

pure nothrow @nogc BitArray opOpAssign(string op)(const BitArray e2)

Constraints:
if (op == "-" || op == "&" || op == "|" || op == "^")

Support for operator op= for BitArray.

pure nothrow BitArray opCatAssign(bool b)
pure nothrow BitArray opCatAssign(BitArray b)

Support for operator ~= for BitArray. Warning: This will overwrite a bit in the final word of the current underlying data regardless of whether it is shared between BitArray objects. i.e. D dynamic array concatenation semantics are not followed

const pure nothrow BitArray opCat(bool b)
const pure nothrow BitArray opCat_r(bool b)
const pure nothrow BitArray opCat(BitArray b)

Support for binary operator ~ for BitArray.

pure nothrow @nogc void opOpAssign(string op)(size_t nbits)

Constraints:
if (op == "<

Operator <<= support.

Shifts all the bits in the array to the left by the given number of bits. The leftmost bits are dropped, and 0's are appended to the end to fill up the vacant bits.

Warning: unused bits in the final word up to the next word boundary may be overwritten by this operation. It does not attempt to preserve bits past the end of the array.

pure nothrow @nogc void opOpAssign(string op)(size_t nbits)

Constraints:
if (op == ">>")

Operator >>= support.

Shifts all the bits in the array to the right by the given number of bits. The rightmost bits are dropped, and 0's are inserted at the back to fill up the vacant bits.

Warning: unused bits in the final word up to the next word boundary may be overwritten by this operation. It does not attempt to preserve bits past the end of the array.

const void toString(scope void delegate(const(char)[]) sink, FormatSpec!char fmt)

Return a string representation of this BitArray.

Two format specifiers are supported:

%s which prints the bits as an array, and

%b which prints the bits as 8-bit byte packets

separated with an underscore.

Examples:

import std.format : format;

debug(bitarray) printf("BitArray.toString unittest\n");
auto b = BitArray([0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1]);

auto s1 = format("%s", b);
writeln(s1); // "[0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1]"

auto s2 = format("%b", b);
writeln(s2); // "00001111_00001111"

const nothrow @property auto bitsSet()

Return a lazy range of the indices of set bits.

Examples:

import std.algorithm.comparison : equal;

auto b1 = BitArray([0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1]);
assert(b1.bitsSet.equal([4, 5, 6, 7, 12, 13, 14, 15]));

BitArray b2;
b2.length = 1000;
b2[333] = true;
b2[666] = true;
b2[999] = true;
assert(b2.bitsSet.equal([333, 666, 999]));

pure nothrow @nogc @safe T swapEndian(T)(T val)

Constraints:
if (isIntegral!T || isSomeChar!T || isBoolean!T)

Swaps the endianness of the given integral value or character.

pure nothrow @nogc @safe auto nativeToBigEndian(T)(T val)

Constraints:
if (canSwapEndianness!T)

Converts the given value from the native endianness to big endian and returns it as a ubyte[n] where n is the size of the given type.

Returning a ubyte[n] helps prevent accidentally using a swapped value as a regular one (and in the case of floating point values, it's necessary, because the FPU will mess up any swapped floating point values. So, you can't actually have swapped floating point values as floating point values).

real is not supported, because its size is implementation-dependent and therefore could vary from machine to machine (which could make it unusable if you tried to transfer it to another machine).

Examples:

int i = 12345;
ubyte[4] swappedI = nativeToBigEndian(i);
writeln(i); // bigEndianToNative!int(swappedI)

double d = 123.45;
ubyte[8] swappedD = nativeToBigEndian(d);
writeln(d); // bigEndianToNative!double(swappedD)

pure nothrow @nogc @safe T bigEndianToNative(T, size_t n)(ubyte[n] val)

Constraints:
if (canSwapEndianness!T && n == T.sizeof)

Converts the given value from big endian to the native endianness and returns it. The value is given as a ubyte[n] where n is the size of the target type. You must give the target type as a template argument, because there are multiple types with the same size and so the type of the argument is not enough to determine the return type.

Taking a ubyte[n] helps prevent accidentally using a swapped value as a regular one (and in the case of floating point values, it's necessary, because the FPU will mess up any swapped floating point values. So, you can't actually have swapped floating point values as floating point values).

Examples:

ushort i = 12345;
ubyte[2] swappedI = nativeToBigEndian(i);
writeln(i); // bigEndianToNative!ushort(swappedI)

dchar c = 'D';
ubyte[4] swappedC = nativeToBigEndian(c);
writeln(c); // bigEndianToNative!dchar(swappedC)

pure nothrow @nogc @safe auto nativeToLittleEndian(T)(T val)

Constraints:
if (canSwapEndianness!T)

Converts the given value from the native endianness to little endian and returns it as a ubyte[n] where n is the size of the given type.

Returning a ubyte[n] helps prevent accidentally using a swapped value as a regular one (and in the case of floating point values, it's necessary, because the FPU will mess up any swapped floating point values. So, you can't actually have swapped floating point values as floating point values).

Examples:

int i = 12345;
ubyte[4] swappedI = nativeToLittleEndian(i);
writeln(i); // littleEndianToNative!int(swappedI)

double d = 123.45;
ubyte[8] swappedD = nativeToLittleEndian(d);
writeln(d); // littleEndianToNative!double(swappedD)

pure nothrow @nogc @safe T littleEndianToNative(T, size_t n)(ubyte[n] val)

Constraints:
if (canSwapEndianness!T && n == T.sizeof)

Converts the given value from little endian to the native endianness and returns it. The value is given as a ubyte[n] where n is the size of the target type. You must give the target type as a template argument, because there are multiple types with the same size and so the type of the argument is not enough to determine the return type.

Taking a ubyte[n] helps prevent accidentally using a swapped value as a regular one (and in the case of floating point values, it's necessary, because the FPU will mess up any swapped floating point values. So, you can't actually have swapped floating point values as floating point values).

real is not supported, because its size is implementation-dependent and therefore could vary from machine to machine (which could make it unusable if you tried to transfer it to another machine).

Examples:

ushort i = 12345;
ubyte[2] swappedI = nativeToLittleEndian(i);
writeln(i); // littleEndianToNative!ushort(swappedI)

dchar c = 'D';
ubyte[4] swappedC = nativeToLittleEndian(c);
writeln(c); // littleEndianToNative!dchar(swappedC)

T peek(T, Endian endianness = Endian.bigEndian, R)(R range)
T peek(T, Endian endianness = Endian.bigEndian, R)(R range, size_t index)
T peek(T, Endian endianness = Endian.bigEndian, R)(R range, size_t* index)

Constraints:
if (canSwapEndianness!T && isForwardRange!R && is(ElementType!R : const(ubyte)))
if (canSwapEndianness!T && isForwardRange!R && hasSlicing!R && is(ElementType!R : const(ubyte)))
if (canSwapEndianness!T && isForwardRange!R && hasSlicing!R && is(ElementType!R : const(ubyte)))

Takes a range of ubytes and converts the first T.sizeof bytes to T. The value returned is converted from the given endianness to the native endianness. The range is not consumed.

Parameters:

T	The integral type to convert the first T.sizeof bytes to.
endianness	The endianness that the bytes are assumed to be in.
R range	The range to read from.
size_t index	The index to start reading from (instead of starting at the front). If index is a pointer, then it is updated to the index after the bytes read. The overloads with index are only available if hasSlicing!R is true.

Examples:

ubyte[] buffer = [1, 5, 22, 9, 44, 255, 8];
writeln(buffer.peek!uint()); // 17110537
writeln(buffer.peek!ushort()); // 261
writeln(buffer.peek!ubyte()); // 1

writeln(buffer.peek!uint(2)); // 369700095
writeln(buffer.peek!ushort(2)); // 5641
writeln(buffer.peek!ubyte(2)); // 22

size_t index = 0;
writeln(buffer.peek!ushort(&index)); // 261
writeln(index); // 2

writeln(buffer.peek!uint(&index)); // 369700095
writeln(index); // 6

writeln(buffer.peek!ubyte(&index)); // 8
writeln(index); // 7

T read(T, Endian endianness = Endian.bigEndian, R)(ref R range)

Constraints:
if (canSwapEndianness!T && isInputRange!R && is(ElementType!R : const(ubyte)))

Takes a range of ubytes and converts the first T.sizeof bytes to T. The value returned is converted from the given endianness to the native endianness. The T.sizeof bytes which are read are consumed from the range.

Parameters:

T	The integral type to convert the first T.sizeof bytes to.
endianness	The endianness that the bytes are assumed to be in.
R range	The range to read from.

Examples:

import std.range.primitives : empty;
ubyte[] buffer = [1, 5, 22, 9, 44, 255, 8];
writeln(buffer.length); // 7

writeln(buffer.read!ushort()); // 261
writeln(buffer.length); // 5

writeln(buffer.read!uint()); // 369700095
writeln(buffer.length); // 1

writeln(buffer.read!ubyte()); // 8
assert(buffer.empty);

void write(T, Endian endianness = Endian.bigEndian, R)(R range, T value, size_t index)
void write(T, Endian endianness = Endian.bigEndian, R)(R range, T value, size_t* index)

Constraints:
if (canSwapEndianness!T && isForwardRange!R && hasSlicing!R && is(ElementType!R : ubyte))
if (canSwapEndianness!T && isForwardRange!R && hasSlicing!R && is(ElementType!R : ubyte))

Takes an integral value, converts it to the given endianness, and writes it to the given range of ubytes as a sequence of T.sizeof ubytes starting at index. hasSlicing!R must be true.

Parameters:

T	The integral type to convert the first T.sizeof bytes to.
endianness	The endianness to write the bytes in.
R range	The range to write to.
T value	The value to write.
size_t index	The index to start writing to. If index is a pointer, then it is updated to the index after the bytes read.

Examples:

{
    ubyte[] buffer = [0, 0, 0, 0, 0, 0, 0, 0];
    buffer.write!uint(29110231u, 0);
    writeln(buffer); // [1, 188, 47, 215, 0, 0, 0, 0]

    buffer.write!ushort(927, 0);
    writeln(buffer); // [3, 159, 47, 215, 0, 0, 0, 0]

    buffer.write!ubyte(42, 0);
    writeln(buffer); // [42, 159, 47, 215, 0, 0, 0, 0]
}

{
    ubyte[] buffer = [0, 0, 0, 0, 0, 0, 0, 0, 0];
    buffer.write!uint(142700095u, 2);
    writeln(buffer); // [0, 0, 8, 129, 110, 63, 0, 0, 0]

    buffer.write!ushort(19839, 2);
    writeln(buffer); // [0, 0, 77, 127, 110, 63, 0, 0, 0]

    buffer.write!ubyte(132, 2);
    writeln(buffer); // [0, 0, 132, 127, 110, 63, 0, 0, 0]
}

{
    ubyte[] buffer = [0, 0, 0, 0, 0, 0, 0, 0];
    size_t index = 0;
    buffer.write!ushort(261, &index);
    writeln(buffer); // [1, 5, 0, 0, 0, 0, 0, 0]
    writeln(index); // 2

    buffer.write!uint(369700095u, &index);
    writeln(buffer); // [1, 5, 22, 9, 44, 255, 0, 0]
    writeln(index); // 6

    buffer.write!ubyte(8, &index);
    writeln(buffer); // [1, 5, 22, 9, 44, 255, 8, 0]
    writeln(index); // 7
}

void append(T, Endian endianness = Endian.bigEndian, R)(R range, T value)

Constraints:
if (canSwapEndianness!T && isOutputRange!(R, ubyte))

Takes an integral value, converts it to the given endianness, and appends it to the given range of ubytes (using put) as a sequence of T.sizeof ubytes starting at index. hasSlicing!R must be true.

Parameters:

T	The integral type to convert the first T.sizeof bytes to.
endianness	The endianness to write the bytes in.
R range	The range to append to.
T value	The value to append.

Examples:

import std.array;
auto buffer = appender!(const ubyte[])();
buffer.append!ushort(261);
writeln(buffer.data); // [1, 5]

buffer.append!uint(369700095u);
writeln(buffer.data); // [1, 5, 22, 9, 44, 255]

buffer.append!ubyte(8);
writeln(buffer.data); // [1, 5, 22, 9, 44, 255, 8]

pure nothrow @nogc auto bitsSet(T)(T value)

Constraints:
if (isIntegral!T)

Range that iterates the indices of the set bits in value. Index 0 corresponds to the least significant bit. For signed integers, the highest index corresponds to the sign bit.

Examples:

import std.algorithm.comparison : equal;
import std.range : iota;

assert(bitsSet(1).equal([0]));
assert(bitsSet(5).equal([0, 2]));
assert(bitsSet(-1).equal(iota(32)));
assert(bitsSet(int.min).equal([31]));