Daylight Savings Time support #2230

deirdreobyrne · 2022-07-04T15:33:12Z

Adds E.setDST(), which gives Espruino rules for determining when daylight savings time starts and ends. Note that E.setTimeZone() will have no effect when daylight savings time rules are in effect. (I note that Gadgetbridge sets the time zone, so by disabling the effectiveness of E.setTimeZone() we prevent clashes between Gadgetbridge and this DST code when DST changes).

Note that one of the tests in tests/test_date.js was actually wrong. It was assuming GMT, when the timezone was explicitly set with E.setTimeZone(2).

DST checks have been added to tests/test_date.js, and all of them are now passing.

src/jswrap_date.c

gfwilliams · 2022-07-05T13:27:23Z

src/jswrap_espruino.c

+
+  if (!jsvIsArray(params)) return;
+  if (jsvGetLength(params) != 12) return;
+  jsvIteratorNew(&it,params,JSIF_DEFINED_ARRAY_ElEMENTS);


Would just replacing the following with jswrap_typedarray_constructor(ARRAYBUFFERVIEW_INT16, params, 0, 0) work here? It'd save a bit of code space

Probably. Still familiarising myself with the data types.

gfwilliams · 2022-07-05T13:28:52Z

src/jswrap_date.c

+}
+
+// Convert a number of days since 1970 into y,m,d. 0<=m<=11
+void getDateFromDayNumber(int day, int *y, int *m, int *date) {


Is there any kind of reference for this that could be linked? This looks a lot like witchcraft to me and it might just be handy to know where it came from.

It is adopted from an algorithm in a book called "Astronomical Algorithms" by Jean Meeus. The book - from 1991 - is well out of print by now.

I'm in the process of doing my own write-up on these algorithms. I've managed to convince myself of the correctness of everything but the constant 2877911. I've also ran getDayNumberFromDate against getDateFromDayNumber, and they are consistent.

Thanks! It's a magic bit of code - very few people doing stuff like that any more :) Just a reference to "Astronomical Algorithms" would be fine but a link to your writeup seems even better ;)

I had to do quite a bit of work to the algorithm in "Astronomical Algorithms", as it had its year zero in the year 4713BC. I wanted my year zero to be 1970 to correspond with the zero of Unix time, and the conversion was quite the mind bend.

I just love algorithms like that - it was a joy to work on!

gfwilliams · 2022-07-05T13:31:28Z

src/jswrap_date.c

+    jsvArrayBufferIteratorNew(&it, dst, 0);
+    y = 0;
+    while (y < 12) {
+      JsVar *setting = jsvArrayBufferIteratorGetValue(&it);


Please could you use dstSetting[y++]=jsvArrayBufferIteratorGetIntegerValue(&it) here? We try and avoid accessing ->varData directly usually (eg using jsvGetInteger) as it makes it easier if we need to tweak the internals.

gfwilliams · 2022-07-05T13:35:41Z

src/jswrap_date.c

+  "type" : "method",
+  "class" : "Date",
+  "name" : "toLocalISOString",
+  "generate" : "jswrap_date_toLocalISOString",


Maybe we could add "ifndef":"SAVE_ON_FLASH" to try and save a few bytes on micro:bit here

gfwilliams · 2022-07-05T13:39:44Z

Hi! This looks really good, thanks! I've just enabled the CI check, and there do seem to be some issues though:

ESP32/Pixl/Pico/MDBT42/etc builds fail because of td.zone = jsdGetTimeZone() in libs/filesystem/jswrap_fs.c (FAT32 filesystem) - it should be a pretty easy fix as I think it's what you're already doing with a call to setCorrectTimeZone(&td);?

Espruino/libs/filesystem/jswrap_fs.c

Line 349 in 042dbff

td.daysSinceEpoch = fromCalenderDate(&date);
micro:bit 1 build fails because this seems to use about 1400 bytes more than the previous code (so adding about 1% to code size). The stuff I suggested may help a little but I'm actually pretty surprised this adds so much - not sure if you have any thoughts here? I guess we could just do #ifndef ESPR_NO_DAYLIGHT_SAVING and add that define on micro:bit 1, which is already a slightly limited build anyway?
Are you using tabs? If you look at jswrap_date.c at 209/318 and other places, the indenting seems a bit off?

But thanks - this looks really neat. It'll be a great addition.

deirdreobyrne · 2022-07-05T14:12:43Z

ESP32/Pixl/Pico/MDBT42/etc builds fail because of td.zone = jsdGetTimeZone() in libs/filesystem/jswrap_fs.c (FAT32 filesystem) - it should be a pretty easy fix as I think it's what you're already doing with a call to setCorrectTimeZone(&td);?

Yes - that should fix the problem

micro:bit 1 build fails because this seems to use about 1400 bytes more than the previous code (so adding about 1% to code size). The stuff I suggested may help a little but I'm actually pretty surprised this adds so much - not sure if you have any thoughts here? I guess we could just do #ifndef ESPR_NO_DAYLIGHT_SAVING and add that define on micro:bit 1, which is already a slightly limited build anyway?

I'm also surprised at the 1400 bytes this takes. If it fails again then I guess micro:bit will have to do without DST.

Are you using tabs? If you look at jswrap_date.c at 209/318 and other places, the indenting seems a bit off?

I've had horrible problems with tabs.

deirdreobyrne · 2022-07-06T10:58:43Z

I've done a bit of refactoring so that the code should take a bit less space. Expect another commit probably later today.

gfwilliams · 2022-07-06T13:14:11Z

Excellent - thanks!

gfwilliams · 2022-07-06T13:14:16Z

Excellent - thanks!

deirdreobyrne · 2022-07-06T14:47:22Z

I think that's about as good as I can make the code. I've implemented the ESPR_NO_DAYLIGHT_SAVING for microbit1, but I can't get it to compile for that platform! Could you review and see if I did ESPR_NO_DAYLIGHT_SAVING correctly? Or am I doing something wrong with my attempts to compile the microbit1 version?

gfwilliams · 2022-07-06T15:24:14Z

Great - thanks! I'll take a look tomorrow and will see about merging

gfwilliams · 2022-07-07T09:33:27Z

This looks great to me - the microbit build seems to work fine as well. Merging now - Thanks for this, and for making those tweaks!

deirdreobyrne · 2022-07-07T10:57:22Z

This looks great to me - the microbit build seems to work fine as well. Merging now - Thanks for this, and for making those tweaks!

Curious. I still cannot get the microbit1 build to work - which means that it's possible the microbit1 could actually support DST. Worth trying it out?

deirdreobyrne · 2022-07-07T11:16:47Z

OK - figured out why I wasn't able to compile for microbit1 - I didn't have RELEASE=1 in my makefile.

The microbit1 cannot run the DST code for want of a measly 332 bytes. So unless I can think of some way of squeezing out another 332 bytes we can call this project done.

gfwilliams · 2022-07-07T12:37:52Z

The microbit1 cannot run the DST code for want of a measly 332 bytes

Well yes, but even if you squeezed it down 332 bytes and got it to compile, it's means that the very next improvement (even if it's a bugfix) will likely break the compile.

microbit 1 is super limited and we leave a lot of stuff out of it already - so I think not having it built in is the right thing to do anyway.

gfwilliams · 2024-01-26T14:52:08Z

Hi! I know this is a while ago, but a segfault issue was just reported at #2456

Some weirdness can be reproduced just by doing d = new Date(-12,8) which seems valid, and this looks like it's down to your replacement getDateFromDayNumber which comes from https://github.com/deirdreobyrne/CalendarAndDST

It's giving negative values for days/months in some cases if the date is before 1970.

Before your changes went in, this worked fine.

// before 
>d = new Date(-12,8)
=Date: Tue Aug 31 -12 00:00:00 GMT+0000

// after
>d = new Date(-12,8).ms
=-62524915200000
>d = new Date(-12,8)
=Date: Wed h -29 -11 00:00:00 GMT+0000

It seemed you had a really thorough write-up in https://github.com/deirdreobyrne/CalendarAndDST so I thought this was worth asking you about first

deirdreobyrne · 2024-01-26T15:03:08Z

Eeeeep - my code is only supposed to be used with the Gregorian calendar (i.e. post-1582, or post-1752 in the UK, US and elsewhere). And yes I can see how it might have a problem pre-1970, as I coded it with the idea that Date() starts in 1970. Which is the most important thing to fix? That it doesn't segfault? That it is consistent going between millis and Dates? Or that it gives what could be considered the "right" answer (which would be for example to use Gregorian after 1582 and Julian before)? D.

…

On Fri, 26 Jan 2024 at 14:52, Gordon Williams ***@***.***> wrote: Hi! I know this is a while ago, but a segfault issue was just reported at #2456 <#2456> Some weirdness can be reproduced just by doing d = new Date(-12,8) which seems valid, and this looks like it's down to your replacement getDateFromDayNumber which comes from https://github.com/deirdreobyrne/CalendarAndDST It's giving negative values for days/months in some cases if the date is before 1970. Before your changes went in, this worked fine. // before >d = new Date(-12,8) =Date: Tue Aug 31 -12 00:00:00 GMT+0000 // after >d = new Date(-12,8).ms =-62524915200000 >d = new Date(-12,8) =Date: Wed h -29 -11 00:00:00 GMT+0000 It seemed you had a really thorough write-up in https://github.com/deirdreobyrne/CalendarAndDST so I thought this was worth asking you about first — Reply to this email directly, view it on GitHub <#2230 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGZJOLEXREUHFVGMCH3QIATYQO7KFAVCNFSM52TSFGA2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJRGIYTSMJSGIZQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

gfwilliams · 2024-01-26T15:13:44Z

Thanks! Well, from my point of view:

It shouldn't ever segfault (eg month always 0..11, day always 1..31)
pre-1970 at minimum (ideally 1582), if you do new Date(-1000,7,1) you should expect it'll say
=Date: ... Jul 1 -1000 00:00:00 GMT+0000 in return. Right now it says Date: Fri ength -27 -999 00:00:00 GMT+0000

I don't think the Date<->millis thing matters pre-1582, but it'd be nice if within reason you could add one day's worth of milliseconds and get to the next day.

deirdreobyrne · 2024-01-27T17:21:58Z

I believe I have a bugfix in my github account. The good news is that I believe it will be good for over 71,000 years. Though in the very distant past and future you might find yourself unable to add or subtract a day, due to the precision of floating point, and you may also have an "incomplete" calendar. HOWEVER in astronomy (which is one of my hobbies) the convention is to use the Gregorian calendar as far back as the Gregorian reform in 1582, and the Julian calendar before that. (And it is also standard practice to have no year zero). My code breaks these two conventions. The solutions I see are - 1) Break the conventions 2) Develop code for the Julian calendar, and code to switch between Julian and Gregorian on the appropriate date in 1582. Whereas different countries implemented the switch on different dates, the convention is to use the 1582 date. 3) Throw some sort of exception for dates before that date in 1582 (if I'm reading your code correctly it's fairly easy to throw exceptions?) Solution 3 is the most technically accurate easy solution, and solution 2 is a reasonably technically accurate difficult solution. I don't like the current state of affairs. What do you think? D.

…

On Fri, 26 Jan 2024 at 15:13, Gordon Williams ***@***.***> wrote: Thanks! Well, from my point of view: - It shouldn't ever segfault (eg month always 0..11, day always 1..31) - pre-1970 at minimum (ideally 1582) I don't really care, if you do new Date(-1000,7,1) you should expect it'll say =Date: ... Jul 1 -1000 00:00:00 GMT+0000 in return. Right now it says Date: Fri ength -27 -999 00:00:00 GMT+0000 I don't think the Date<->millis thing matters pre-1582, but it'd be nice if within reason you could add one day's worth of milliseconds and get to the next day. — Reply to this email directly, view it on GitHub <#2230 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGZJOLFXEB6YHNGEDA3EJEDYQPB3FAVCNFSM52TSFGA2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJRGIZDEOBYHAYQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

gfwilliams · 2024-01-29T14:42:29Z

Sorry for the delayed response - what you've managed with #2459 looks great though!

Having as wider range as possible using Gregorian and throwing an exception outside of that seems great

Bugfix for #2456 (PR #2230) - Date library bug fixes

deirdreobyrne added 17 commits June 22, 2022 16:06

Well tested C code for dates and DST

615e4dc

Merge branch 'espruino:master' into DST

32b4db8

Outline of the challenge.

5f9567b

E.setDST and E.setTimeZone

5493f1d

Start of date handling code

4ff86bd

Should now be able to determine whether DST active

67ce31c

Much of the code written

f4ffcea

Update jswrap_date.c

81a57e5

Now compiles

283b04e

Tabs to spaces, starting debugging

d60896c

bugfixes

3d6ddf4

tabs to spaces, and bug fix

58b82b6

Incorrect test case! Assumed GMT rather than local

2f3e56f

Bug fix, and added tests for DST code

301dbfd

Adding tests for 0.1s either side of DST changes

2454adf

Correcting a comment - got seasons backwards

71f0d09

Southern hemisphere tests revealed a bug!

6822cf7

deirdreobyrne commented Jul 5, 2022

View reviewed changes

src/jswrap_date.c Outdated Show resolved Hide resolved

gfwilliams reviewed Jul 5, 2022

View reviewed changes

Fixes arising from code review

333cb9b

Merge branch 'espruino:master' into DST

d453b14

Code efficiencies added, but microbit1 still fails

1e7b72a

gfwilliams merged commit 450aec5 into espruino:master Jul 7, 2022

deirdreobyrne deleted the DST branch July 7, 2022 11:30

gfwilliams mentioned this pull request Jan 26, 2024

SEGV on unknown address in jsvStringIteratorPrintfCallback #2456

Closed

gfwilliams added a commit that referenced this pull request Jan 29, 2024

Merge pull request #2459 from deirdreobyrne/date_limit

fcd68f6

Bugfix for #2456 (PR #2230) - Date library bug fixes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Daylight Savings Time support #2230

Daylight Savings Time support #2230

deirdreobyrne commented Jul 4, 2022 •

edited

Loading

gfwilliams Jul 5, 2022

deirdreobyrne Jul 5, 2022

gfwilliams Jul 5, 2022

deirdreobyrne Jul 5, 2022

gfwilliams Jul 5, 2022

deirdreobyrne Jul 5, 2022

gfwilliams Jul 5, 2022

deirdreobyrne Jul 5, 2022

gfwilliams Jul 5, 2022

deirdreobyrne Jul 5, 2022

gfwilliams commented Jul 5, 2022

deirdreobyrne commented Jul 5, 2022

deirdreobyrne commented Jul 6, 2022

gfwilliams commented Jul 6, 2022

gfwilliams commented Jul 6, 2022

deirdreobyrne commented Jul 6, 2022

gfwilliams commented Jul 6, 2022

gfwilliams commented Jul 7, 2022

deirdreobyrne commented Jul 7, 2022

deirdreobyrne commented Jul 7, 2022

gfwilliams commented Jul 7, 2022

gfwilliams commented Jan 26, 2024

deirdreobyrne commented Jan 26, 2024 via email

gfwilliams commented Jan 26, 2024 •

edited

Loading

deirdreobyrne commented Jan 27, 2024 via email

gfwilliams commented Jan 29, 2024

Daylight Savings Time support #2230

Daylight Savings Time support #2230

Conversation

deirdreobyrne commented Jul 4, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gfwilliams commented Jul 5, 2022

deirdreobyrne commented Jul 5, 2022

deirdreobyrne commented Jul 6, 2022

gfwilliams commented Jul 6, 2022

gfwilliams commented Jul 6, 2022

deirdreobyrne commented Jul 6, 2022

gfwilliams commented Jul 6, 2022

gfwilliams commented Jul 7, 2022

deirdreobyrne commented Jul 7, 2022

deirdreobyrne commented Jul 7, 2022

gfwilliams commented Jul 7, 2022

gfwilliams commented Jan 26, 2024

deirdreobyrne commented Jan 26, 2024 via email

gfwilliams commented Jan 26, 2024 • edited Loading

deirdreobyrne commented Jan 27, 2024 via email

gfwilliams commented Jan 29, 2024

deirdreobyrne commented Jul 4, 2022 •

edited

Loading

gfwilliams commented Jan 26, 2024 •

edited

Loading