vsl_binary_explicit_io.h File Reference

Byte-swapping, arbitrary length integer conversion, and explicit I/O. More...

#include <cstring>
#include <cstddef>
#include <iostream>
#include <vxl_config.h>
#include <cassert>
#include "vsl_binary_io.h"

Go to the source code of this file.

Macros
#define	VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(size_of_type)   ((((size_of_type) * 8)/7) + (((((size_of_type) * 8) % 7) == 0) ? 0: 1))
	The maximum length of buffer to use with arbitrary length integers. More...

Functions
void	vsl_swap_bytes (char *ptr, unsigned nbyte, std::size_t nelem=1)
	Perform byte swapping in situ. More...
void	vsl_swap_bytes_to_buffer (const char *source, char *dest, unsigned nbyte, std::size_t nelem=1)
	Perform byte swapping to a buffer. More...
template<class T >
std::size_t	vsl_convert_to_arbitrary_length_unsigned_impl (const T *ints, unsigned char *buffer, std::size_t count)
	Implement arbitrary length conversion for unsigned integers. More...
template<class T >
std::size_t	vsl_convert_to_arbitrary_length_signed_impl (const T *ints, unsigned char *buffer, std::size_t count)
	Implement arbitrary length conversion for signed integers. More...
template<class T >
std::size_t	vsl_convert_from_arbitrary_length_signed_impl (const unsigned char *buffer, T *ints, std::size_t count)
	Implement arbitrary length conversion for signed integers. More...
template<class T >
std::size_t	vsl_convert_from_arbitrary_length_unsigned_impl (const unsigned char *buffer, T *ints, std::size_t count=1)
	Implement arbitrary length conversion for unsigned integers. More...
std::size_t	vsl_convert_to_arbitrary_length (const unsigned long *ints, unsigned char *buffer, std::size_t count=1)
	Encode an array of ints into an arbitrary length format. More...
std::size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, unsigned long *ints, std::size_t count=1)
	Decode a buffer of arbitrary length integers. More...
std::size_t	vsl_convert_to_arbitrary_length (const long *ints, unsigned char *buffer, std::size_t count=1)
	Encode an array of ints into an arbitrary length format. More...
std::size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, long *ints, std::size_t count=1)
	Decode a buffer of arbitrary length integers. More...
std::size_t	vsl_convert_to_arbitrary_length (const unsigned int *ints, unsigned char *buffer, std::size_t count=1)
	Encode an array of ints into an arbitrary length format. More...
std::size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, unsigned int *ints, std::size_t count=1)
	Decode a buffer of arbitrary length integers. More...
std::size_t	vsl_convert_to_arbitrary_length (const int *ints, unsigned char *buffer, std::size_t count=1)
	Encode an array of ints into an arbitrary length format. More...
std::size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, int *ints, std::size_t count=1)
	Decode a buffer of arbitrary length integers. More...
std::size_t	vsl_convert_to_arbitrary_length (const unsigned short *ints, unsigned char *buffer, std::size_t count=1)
	Encode an array of ints into an arbitrary length format. More...
std::size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, unsigned short *ints, std::size_t count=1)
	Decode a buffer of arbitrary length integers. More...
std::size_t	vsl_convert_to_arbitrary_length (const short *ints, unsigned char *buffer, std::size_t count=1)
	Encode an array of ints into an arbitrary length format. More...
std::size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, short *ints, std::size_t count=1)
	Decode a buffer of arbitrary length integers. More...
void	vsl_b_write_uint_16 (vsl_b_ostream &os, unsigned long n)
	Write an unsigned int as 16 bits to vsl_b_ostream. More...
void	vsl_b_read_uint_16 (vsl_b_istream &is, unsigned long &n)
	Read an unsigned int as 16 bits from vsl_b_istream. More...
void	vsl_b_write_int_16 (vsl_b_ostream &os, long n)
	Write a signed int as 16 bits to vsl_b_ostream. More...
void	vsl_b_read_int_16 (vsl_b_istream &is, long &n)
	Read a signed int as 16 bits from vsl_b_istream. More...
void	vsl_b_write_uint_64 (vsl_b_ostream &os, std::size_t n)
	Write a std::size_t as 64 bits to vsl_b_ostream. More...
void	vsl_b_read_uint_64 (vsl_b_istream &is, std::size_t &n)
	Read a std::size_t as 64 bits from vsl_b_istream. More...

Variables
There is a problem with the ENDIAN indication	macros

Detailed Description

Byte-swapping, arbitrary length integer conversion, and explicit I/O.

Author

Ian Scott (Manchester) April 2001

Include this file if you want to perform integer IO using fixed size encoding.

If you want to read or write a large number of floating points, then;

• Floats and doubles need byte swapped, and this can be done in situ.
• Shorts, ints and longs need converted to/from the arbitrary length format.

Implementation details:

The arbitrary length encoding takes the number and breaks it into 7 bit nibbles. Each nibble is saved with the 8th bit set if this is the last byte. The nibbles are stored in little endian order. ie The first byte of the encoded format represents the least significant 7 bits.

The algorithm used to encode an unsigned value works as follows

  while value is greater than 2^7-1
    emit 0 bit
    emit least significant 7 bits of value.
    shift value right 7 bits
  emit bit 1
  emit value embedded in 7 bits

Definition in file vsl_binary_explicit_io.h.

Macro Definition Documentation

VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH

#define VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH

(

size_of_type

)

((((size_of_type) * 8)/7) + (((((size_of_type) * 8) % 7) == 0) ? 0: 1))

The maximum length of buffer to use with arbitrary length integers.

This macro tells you the size of buffer you need when using vsl_convert_ints_to_arbitrary_length(). You should give the macro the size of the type you want to convert. If you are converting several integers at once, multiply the value by the number of integers.

Definition at line 163 of file vsl_binary_explicit_io.h.

Function Documentation

vsl_b_read_int_16()

void vsl_b_read_int_16 ( vsl_b_istream &  is, long &  n  )

inline

Read a signed int as 16 bits from vsl_b_istream.

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

This function will only read values saved using vsl_b_write_int_16().

Definition at line 668 of file vsl_binary_explicit_io.h.

vsl_b_read_uint_16()

void vsl_b_read_uint_16 ( vsl_b_istream &  is, unsigned long &  n  )

inline

Read an unsigned int as 16 bits from vsl_b_istream.

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

This function will only read values saved using vsl_b_write_uint_16().

Definition at line 635 of file vsl_binary_explicit_io.h.

vsl_b_read_uint_64()

void vsl_b_read_uint_64 ( vsl_b_istream &  is, std::size_t &  n  )

inline

Read a std::size_t as 64 bits from vsl_b_istream.

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

This function will only read values saved using vsl_b_write_uint_64().

Definition at line 708 of file vsl_binary_explicit_io.h.

vsl_b_write_int_16()

void vsl_b_write_int_16 ( vsl_b_ostream &  os, long  n  )

inline

Write a signed int as 16 bits to vsl_b_ostream.

If your signed int cannot be represented in 16 bits (e.g. on a 32 bit platform) the stream's error flag will be set.

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

You must vsl_b_read_int_16() to read the value saved with this function.

Definition at line 654 of file vsl_binary_explicit_io.h.

vsl_b_write_uint_16()

void vsl_b_write_uint_16 ( vsl_b_ostream &  os, unsigned long  n  )

inline

Write an unsigned int as 16 bits to vsl_b_ostream.

If your signed int cannot be represented in 16 bits (e.g. on a 32 bit platform) the stream's error flag will be set.

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

You must use vsl_b_read_uint_16() to read the value saved with this function.

Definition at line 621 of file vsl_binary_explicit_io.h.

vsl_b_write_uint_64()

void vsl_b_write_uint_64 ( vsl_b_ostream &  os, std::size_t  n  )

inline

Write a std::size_t as 64 bits to vsl_b_ostream.

Will assert if your std::size_t cannot be represented in 64 bits (e.g. on some 128 bit platforms).

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

You must use vsl_b_read_uint_64() to read the value saved with this function.

Definition at line 694 of file vsl_binary_explicit_io.h.

vsl_convert_from_arbitrary_length() [1/6]

std::size_t vsl_convert_from_arbitrary_length ( const unsigned char *  buffer, unsigned long *  ints, std::size_t  count = 1  )

inline

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal longs.

Parameters

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns

the number of bytes used, or zero on error.

Definition at line 326 of file vsl_binary_explicit_io.h.

vsl_convert_from_arbitrary_length() [2/6]

std::size_t vsl_convert_from_arbitrary_length ( const unsigned char *  buffer, long *  ints, std::size_t  count = 1  )

inline

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal longs.

Parameters

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns

the number of bytes used, or zero on error.

Definition at line 356 of file vsl_binary_explicit_io.h.

vsl_convert_from_arbitrary_length() [3/6]

std::size_t vsl_convert_from_arbitrary_length ( const unsigned char *  buffer, unsigned int *  ints, std::size_t  count = 1  )

inline

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal ints.

Parameters

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns

the number of bytes used, or zero on error.

Definition at line 387 of file vsl_binary_explicit_io.h.

vsl_convert_from_arbitrary_length() [4/6]

std::size_t vsl_convert_from_arbitrary_length ( const unsigned char *  buffer, int *  ints, std::size_t  count = 1  )

inline

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal ints.

Parameters

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns

the number of bytes used, or zero on error.

Definition at line 418 of file vsl_binary_explicit_io.h.

vsl_convert_from_arbitrary_length() [5/6]

std::size_t vsl_convert_from_arbitrary_length ( const unsigned char *  buffer, unsigned short *  ints, std::size_t  count = 1  )

inline

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal ints.

Parameters

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns

the number of bytes used, or zero on error.

Definition at line 449 of file vsl_binary_explicit_io.h.

vsl_convert_from_arbitrary_length() [6/6]

std::size_t vsl_convert_from_arbitrary_length ( const unsigned char *  buffer, short *  ints, std::size_t  count = 1  )

inline

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal ints.

Parameters

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns

the number of bytes used, or zero on error.

Definition at line 480 of file vsl_binary_explicit_io.h.

vsl_convert_from_arbitrary_length_signed_impl()

template<class T >

std::size_t vsl_convert_from_arbitrary_length_signed_impl ( const unsigned char *  buffer, T *  ints, std::size_t  count  )

inline

Implement arbitrary length conversion for signed integers.

This function should only be used by this header file.

Definition at line 216 of file vsl_binary_explicit_io.h.

vsl_convert_from_arbitrary_length_unsigned_impl()

template<class T >

std::size_t vsl_convert_from_arbitrary_length_unsigned_impl ( const unsigned char *  buffer, T *  ints, std::size_t  count = 1  )

inline

Implement arbitrary length conversion for unsigned integers.

This function should only be used by this header file.

Definition at line 265 of file vsl_binary_explicit_io.h.

vsl_convert_to_arbitrary_length() [1/6]

std::size_t vsl_convert_to_arbitrary_length ( const unsigned long *  ints, unsigned char *  buffer, std::size_t  count = 1  )

inline

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(unsigned long)) * count

Definition at line 310 of file vsl_binary_explicit_io.h.

vsl_convert_to_arbitrary_length() [2/6]

std::size_t vsl_convert_to_arbitrary_length ( const long *  ints, unsigned char *  buffer, std::size_t  count = 1  )

inline

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(long)) * count

Definition at line 340 of file vsl_binary_explicit_io.h.

vsl_convert_to_arbitrary_length() [3/6]

std::size_t vsl_convert_to_arbitrary_length ( const unsigned int *  ints, unsigned char *  buffer, std::size_t  count = 1  )

inline

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(unsigned int)) * count

Definition at line 370 of file vsl_binary_explicit_io.h.

vsl_convert_to_arbitrary_length() [4/6]

std::size_t vsl_convert_to_arbitrary_length ( const int *  ints, unsigned char *  buffer, std::size_t  count = 1  )

inline

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(int)) * count

Definition at line 401 of file vsl_binary_explicit_io.h.

vsl_convert_to_arbitrary_length() [5/6]

std::size_t vsl_convert_to_arbitrary_length ( const unsigned short *  ints, unsigned char *  buffer, std::size_t  count = 1  )

inline

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(unsigned short)) * count

Definition at line 432 of file vsl_binary_explicit_io.h.

vsl_convert_to_arbitrary_length() [6/6]

std::size_t vsl_convert_to_arbitrary_length ( const short *  ints, unsigned char *  buffer, std::size_t  count = 1  )

inline

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(short)) * count

Definition at line 463 of file vsl_binary_explicit_io.h.

vsl_convert_to_arbitrary_length_signed_impl()

template<class T >

std::size_t vsl_convert_to_arbitrary_length_signed_impl ( const T *  ints, unsigned char *  buffer, std::size_t  count  )

inline

Implement arbitrary length conversion for signed integers.

This function should only be used by this header file. Returns the number of bytes written

Definition at line 194 of file vsl_binary_explicit_io.h.

vsl_convert_to_arbitrary_length_unsigned_impl()

template<class T >

std::size_t vsl_convert_to_arbitrary_length_unsigned_impl ( const T *  ints, unsigned char *  buffer, std::size_t  count  )

inline

Implement arbitrary length conversion for unsigned integers.

This function should only be used by this header file. Returns the number of bytes written

Definition at line 171 of file vsl_binary_explicit_io.h.

vsl_swap_bytes()

void vsl_swap_bytes ( char *  ptr, unsigned  nbyte, std::size_t  nelem = 1  )

inline

Perform byte swapping in situ.

Where appropriate, swaps pairs of bytes (behaviour is system dependent) Apply this function to your floating-point data to convert from system format to I/O format. Apply the same function to do the reverse conversion.

Parameters

ptr	The buffer to be byte-swapped.
nbyte	The length of the fundamental type, e.g. sizeof(float).
nelem	The number of elements in the buffer (default: 1).

The standard I/O format is little-endian. The code assumes that your system's floats and doubles are stored in memory in either little-endian or big-endian IEEE floating point formats.

Note: There is no point in #ifdef-ing out calls to byte-swapping if you are on a little-endian machine. An optimising compiler will inline the function to nothing for little-endian machines anyway.

If your computer doesn't use IEEE format reals, then we really should redesign the floating point IO. Proposed design notes: Should do conversion to and from a buffer, rather than in place, (since size not known in general) double and reals should be converted to IEEE format. Someone needs to write a long double format anyway. Don't forget to fix all the code that calls vsl_swap_bytes. Really should check anything that #includes this file.

Definition at line 83 of file vsl_binary_explicit_io.h.

vsl_swap_bytes_to_buffer()

void vsl_swap_bytes_to_buffer ( const char *  source, char *  dest, unsigned  nbyte, std::size_t  nelem = 1  )

inline

Perform byte swapping to a buffer.

Same as vsl_swap_bytes, but saves the results in a buffer. In general use vsl_swap_bytes where possible, because it is faster.

Definition at line 108 of file vsl_binary_explicit_io.h.

Variable Documentation

macros

There is a problem with the ENDIAN indication macros

Definition at line 47 of file vsl_binary_explicit_io.h.