This manual documents the dateutils package.

Copyright © 2011-2016 Sebastian Freundt.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts.  A copy of the license is included in the section entitled "GNU Free Documentation License".

dateutils
*********

dateutils are a bunch of tools that revolve around fiddling with dates and times in the command line with a strong focus on use cases that arise when dealing with large amounts of financial data.

Introduction
************

The dateutils package is an attempt to provide a comprehensive set of command line tools that behave intuitively, support a similar variety of inputs, allow for normalised output and, most importantly, cover nearly all tasks that arise whilst dealing with huge amounts of dates and unusual calendars.

In day to day business, the authors have run across a plethora of approaches to tackle seemingly simple problems.  When the command line is a necessity most of these involve perl or awk, which suffer from bad maintainability simply because those tools weren't made to process dates and times in particular, and for the same reason those approaches are anything but fast.

And conversely, good maintainability and fast processing is often seen in integrated systems, such as R or database solutions but they come at a price: little flexibility.  It's very uncommon to import billions of dates into a database just to compare if they are greater than or equal to a given date.  Besides, most of the time those billions of dates come from external sources and have to be brought into shape before or whilst importing them into an integrated system.

Calendars
*********

All of the dateutils command line apps are built around a library, libdut.  The textual representation of dates is converted to an internal one first, then the date operation is performed and the internal representation in the end is converted back to a textual one.

While many of the tools that provide date arithmetic or conversions employ a single standard internal representation (mostly days since a reference date, or seconds since the epoch), the date-core library provides a variety of calendars, each suitable for one or more specific tasks.

Usually it's the tool's task to find out which calendar is the best for the task at hand, but to use the library directly we give an overview of the supported calendars here along with some of their properties.

ymd, aka gregorian
==================

This is the `ordinary' calendar, holding a year, a month and the day of the month.  Years are subdivided into 12 months, months are subdivided into 28 to 31 days.

The canonical input and output format is %Y-%m-%d (ISO 8601). For outputs the format string ymd can be used as well.

ymcw
====

This is a calendric system used at some security exchanges, and can be thought of as a mixture of the year-month-day and year-week-day calendars.  It holds a year, a month, a weekday and the count of the weekday within the month.  So dates like the third Thursday in March 2011 can be expressed trivially in this system.  Years are subdivided into months as in the ymd calendar, months consist of at least 4 and up to 5 weeks and weeks are subdivided into weekdays, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday and Sunday.

The canonical input and output format is %Y-%m-%c-%w.  The week is enumerated by 01 for Monday, 02 for Tuesday, ..., 07 for Sunday, as mandated by ISO 8601.  For outputs the format string ymcw can be used as well.

ywd
===

This calendar is ISO's 8601 year-week-day calendar.  Every year is divided into 52 weeks.  Every week is divided into 7 days, starting with Monday.  To make up for ``missing days'' a leap week is introduced every now and again, i.e. years with leap weeks are 53 weeks long and called leap years (note: ISO does not use the term leap year or leap week).

The canonical input and output format is %rY-W%V-%u, so e.g. 2012-W04-02.  Note that ywd years don't necessarily correspond to Gregorian/ymd years, hence the r modifier in the spec (%rY is used in calendars to select the ``real'' calendar year). The format string ywd can be used as well.

daisy
=====

This is one of the most common calendric systems, it counts the days since a given reference date (we originally went for daysi, short for days since, but after the 100th spelling mistake, because daisy is so much more natural, we decided to stick with daisy).  In the Julian calendar this was the founding of Rome, in many database systems this is the 1st Jan 1, or other convenient dates like the 1st Jan 1970 or 1st Jan 1900.

The (internal) dateutils reference day is the 31st Dec 1600, or equivalently the Naught-th of Jan 1601.  This may look like an arbitrary date but it has some interesting properties: - it is a Sunday, so weekday computation is easy as it is a mod 7 operation only - the year is 1 mod 4, so the number of leap years between the reference year and any other year Y is simply Y div 4.

In earlier versions we used 1917 for the reference year because it's the first year after 1900 that meets the conditions above.  But then we relaxed leap year rules to support dates beyond 2100 (2100 isn't a leap year) so for symmetry reasons we should support years before 1917 in a similar fashion.

The daisy calendar has no notion of years or months.

The daisy calendar as such has no associated input or output format. This is because we reserve the freedom to change the reference date in the future (like we have done in the past).  However, daisy is easily converted to and from other similar day counting systems so their input and output format may be used instead.

Also noteworthy on a more informative level, dateutils globally assumes the Gregorian date switch to have taken place in October 1582.  This is unlike some other well-known tools (Unix cal for instance) which assume the reformation to have occurred in September 1752.

julian, jdn
-----------

A close sibling to the daisy system described before is the Julian day number, with reference date 01 Jan 4713 BC in the proleptic Julian calendar.

The input and output format string is jdn or julian, there is no format specifier.

lilian, ldn
-----------

Another close sibling to the daisy system is the Lilian day number, its reference date is the first day of the Gregorian calendar, 15 Oct 1582.

The input and output format string is ldn or lilian, there is no format specifier.

matlab, mdn
-----------

Taking the Lilian day number further (in a proleptic sense) to the beginning of time, the First of January in year 0, you get the Matlab day number.

The input and output format string is mdn or matlab, there is no format specifier.

bizda
=====

This calendric system is actually a whole tribe of calendars, in its canonical form this tracks the number of business days after last month's ultimo.  Where business day means any non-weekend day.  Years are divided into 12 months, each of which consists of 20 to 23 business days.

The canonical input and output format is %Y-%m-%db, where b is a suffix modifier meaning business days since last month's ultimo.  Similarly, %Y-%m-%dB stands for the business days until this month's ultimo.

Format specifiers and units
***************************

The tools provided by dateutils would be pretty meaningless if you could not control how input is to be read or results are to be printed. Following the footsteps of proven systems (such as libc) output formatting is controlled through format specifiers (and flags), as covered by the next 2 sections.

Nonetheless, some tools (dadd, dseq) require durations to be input.  Their syntax is covered in the final section.

     [file format.texi not included]
     [file format-ddiff.texi not included]
     [file units.texi not included]

Localised input and output
**************************

Input and output may need to be refined even beyond what simple format specifiers can accomplish, for example time zone information might not be present in the input (or must not be present in the output); or using named weekdays (%a, %A) or months (%b, %B) depending on region and language.

We present dateutils' solution in the next 2 sections.

     [file zone.texi not included]
     [file locale.texi not included]

Algorithms
**********

Coming from a financial background, shaving milliseconds off of code is daily core business.  And that in turn is why some of the algorithms used in dateutils might look different to what other people do.

In this chapter key concepts are briefly introduced.  For implementation details see the *-core.c and *-core.h files.

32-adic year-days
=================

Months, in the Gregorian sense, are roughly 30 days long.  Some are longer, one is shorter.  It comes in handy though that the average month length is close to a 2-power, 32.  So we can introduce a 32-adic year as follows:

year-day        traditional     32-adic m/d
0       19      Jan|00          00|19
1       20      Jan|01          00|20
31      50      Jan|31          01|18

32      51      Feb|01          01|19
59      78      Feb|28          02|14

60      79      Mar|01          02|15
90      109     Mar|31          03|13

91      110     Apr|01          03|14
120     139     Apr|30          04|11

121     140     May|01          04|12
151     170     May|31          05|10

181     200     Jun|30          06|08
212     231     Jul|31          07|07
243     262     Aug|31          08|06
273     292     Sep|30          09|04
304     323     Oct|31          10|03
334     353     Nov|30          11|01
365     384     Dec|31          12|00

As can be seen, the year has 19 unexplained days at the beginning, and, more unintuitively, (traditional) months start in one (32-adic) month and end in another.  But the ending always coincides with the traditional months.

Another nice thing about this year is that all months end on a different day; this is nice because you can deduce the month just by naming its ending (or the beginning respectively, as all months as a consequence also start on a different day).

The final nice thing lies in the property, that months that end (or start) on a day <=16, are affected by the leap day introduced in Feb in the traditional Gregorian calendar.  By affected we mean, that the months start (and end) one day off in a leap year.

So to cut this story short, the 32-adic year makes computations easier that need a lot of mapping between the day in the year and the month+day representation, e.g. weekday computations (the 32-adic year preserves weekdays as 32 and 7 are co-prime), day differences, etc.

The algorithm to get the day of the year from a 32-adic month+day is:
Input: m <- 32-adic month
       d <- 32-adic day of month
Output: yd -> day of year

yd <- m * 32 + d - 19

And conversely.  The algorithm to get the 32-adic month+day is trivial.

To convert between 32-adic month+day and Gregorian month+day:
Input: m <- 32-adic month
       d <- 32-adic day of month
Output: m -> 32-adic month
        d -> 32-adic day of month

/* day of month of the beginning and end of m */
bom <- get_beg_of_month(m)
eom <- get_end_of_month(m)

/* get yd as described above */
yd <- get_yd(m, d)

if d <= eom
        d <- yd - (32 * (m - 1) - 19 + bom - 1)
else
        d <- yd - (32 * m - 19 + eom)
        m <- m + 1

     [file dateadd.texi not included]
     [file dateconv.texi not included]
     [file datediff.texi not included]
     [file dategrep.texi not included]
     [file dateround.texi not included]
     [file dateseq.texi not included]
     [file datesort.texi not included]
     [file datetest.texi not included]
     [file datezone.texi not included]
     [file strptime.texi not included]