By the astronomic numbers, yesterday afternoon was the shortest afternoon.
Starting today, the afternoons begin, ever so slowly, to regain light. By mid-January you can already see the difference as you go home from work or milk the cows. This is illustrated here:
Go to the last section, Other Phenomena, for an explanation.
Sorry, the mornings continue to lose light until early January.
The shortest day is but an average of mornings and afternoons. Analemma is the answer. A Farmer's Almanac will reveal the numbers.