COMING SOON: JSON parser for N:

Moderatoren: Sleeπ, andymanone

Antworten
tschak909
Beiträge: 197
Registriert: 17.08.2021 00:22
Has thanked: 4 times
Been thanked: 131 times
Kontaktdaten:

COMING SOON: JSON parser for N:

Beitrag von tschak909 »

Hello everyone,

A year ago, I had to rewrite the network code. It meant that the JSON parser that I had originally written, was taken out, until I could replace it.

I have now done the first pass of the JSON parser, which allows one to:

* Open a connection to some JSON data, e.g. (N:HTTPS://oldbytes.space/api/v1/timelines/public?limit=1)
* Set the channel mode to JSON (XIO 252,#1,12,1,"N:")
* Tell the JSON parser to parse that information (XIO ASC("P),#1,12,0,"N:")

And from this point, the JSON document is parsed in the FujiNet, and can be queried for any pieces of information you wish to retrieve, e.g.

Code: Alles auswählen

XIO ASC("Q"),#1,12,0,"N:/0/account/display_name"
And it will be returned via the I/O channel, ready to be picked up with a CIO GET CHARACTER or GET RECORD call e.g.

Code: Alles auswählen

INPUT #1,NAME$
The current JSON parser will parse all of the JSON primitives.

An example of this, is the world's simplest Mastodon client, written in ATARI BASIC, which displays the newest public post:

Code: Alles auswählen

0 DIM A$(256)
10 OPEN #1,12,0,"N:HTTPS://oldbytes.space/api/v1/timelines/public?limit=1"
11 TRAP 91
20 XIO 252,#1,12,1,"N:":REM  SET CHANNEL MODE
30 XIO ASC("P"),#1,12,0,"N:":REM PARSE JSON
40 XIO ASC("Q"),#1,12,0,"N:/0/created_at"
50 INPUT #1,A$:? A$
60 XIO ASC("Q"),#1,12,0,"N:/0/account/display_name"
70 INPUT #!,A$:? A$
80 XIO ASC("Q"),#1,12,0,"N:/0/content/"
90 GET #1,A:? CHR$(A);:GOTO 90
91 CLOSE #1:END
Yes, really, that's it.

And you get something like this:
20220515_110504.jpg
It will be in the next released build for both Atari and Coleco Adam, which will be released very soon.

-Thom

Benutzeravatar
hermann
Beiträge: 37
Registriert: 17.08.2021 07:54
Has thanked: 19 times
Been thanked: 8 times
Kontaktdaten:

Re: COMING SOON: JSON parser for N:

Beitrag von hermann »

Hello Thom,

That’s great to read.
Maybe you remember, a few months ago I wanted to control my Philips Hue Bridge from the Atari after I got my FujiNet device and you told me, that the JSON parser wasn’t in anymore. Doh!
Seems I can give that a try soon again.

-Hermann

tschak909
Beiträge: 197
Registriert: 17.08.2021 00:22
Has thanked: 4 times
Been thanked: 131 times
Kontaktdaten:

Re: COMING SOON: JSON parser for N:

Beitrag von tschak909 »

It has been folded into the latest firmware build.

I really do hope people actually start writing software, am getting very burned out writing 99% of it. :)

-Thom

dabada79
Beiträge: 4
Registriert: 16.02.2022 21:15
Has thanked: 7 times
Been thanked: 1 time
Kontaktdaten:

JSON parser and big numbers

Beitrag von dabada79 »

Hello Thom,

the JSON functionality is amazing. In the past I tried to write some programs doing stuff with the OpenStreetMap API which offers XML and JSON but parsing it alone is a hassle. With the FujiNet this could be a lot easier. As I tested some queries I noted that the type detection (number, string etc.) seems to be hard coded and for number > 2^32 this is a problem as I could not find a way to get the correct numbers. Let me show in code:

Code: Alles auswählen

0 DIM A$(256)
10 OPEN #1,12,0,"N2:https://www.openstreetmap.org/api/0.6/way/36728867.json"
11 TRAP 91
20 XIO 252,#1,12,1,"N2:"
30 XIO ASC("P"),#1,12,0,"N2:"
40 XIO ASC("Q"),#1,12,0,"N2:/elements/0/tags/name"
45 INPUT #1,A$:? A$
50 XIO ASC("Q"),#1,12,0,"N2:/elements/0/nodes"
55 INPUT #1,A$:? A$:REM NEITHER A NOR A$ WORKS
57 GOTO 55
91 CLOSE #1:END 
Besides, I am using N2 here because I am using NOS and N1 is in use by the NCD mount to my TNFS server where the Basic files are. Took me a while to figure out that using N: for JSON stuff while also using N: as mounted drive leads to an ERROR 170
When you run the program you will get this:

Code: Alles auswählen

LiliensteinstraCe
559468538
2147483647
2147483647
1629498812
1629498816
271661494
2147483647
1149629891
2147483647
88152499
271661531
2147483647
2147483647
271661530
2147483647
2147483647
271661522
2147483647
271661527
2147483647
271661526
The first word is a German street name (Liliensteinstraße) therefore the scrambled UTF-8 character - not a problem for now. The numbers following the street name are the problem. The first one is correct because it is in range. The second one should be 5091761612 but printed is 2147483647 as are all numbers that do not fit into signed 31 bit.
A short peek into the source code (./fujinet-platformio/lib/fnjson/fnjson.cpp, around line 108) lets me think the firmware has to be changed but maybe there is a way to disable the type detection user controlled (i.e. SIO parameter DAUX2) so the JSON parser returns just the values as string instead?

But maybe I'm missing something completely here.

Nevertheless, nice work!

Greetings, Daniel

tschak909
Beiträge: 197
Registriert: 17.08.2021 00:22
Has thanked: 4 times
Been thanked: 131 times
Kontaktdaten:

Re: COMING SOON: JSON parser for N:

Beitrag von tschak909 »

Thank you for this test case!

A bug report has been filed:
https://github.com/FujiNetWIFI/fujinet- ... issues/512

and as it turns out, the fix was easy enough.
https://github.com/FujiNetWIFI/fujinet- ... 7451fc7267

So I will ask @mozzwald to spin another atari build when checks pass.

-Thom

tschak909
Beiträge: 197
Registriert: 17.08.2021 00:22
Has thanked: 4 times
Been thanked: 131 times
Kontaktdaten:

Re: COMING SOON: JSON parser for N:

Beitrag von tschak909 »

The firmware has now been updated, and you can update your FujiNet via the flasher to get the fix.

-Thom

dabada79
Beiträge: 4
Registriert: 16.02.2022 21:15
Has thanked: 7 times
Been thanked: 1 time
Kontaktdaten:

Re: COMING SOON: JSON parser for N:

Beitrag von dabada79 »

tschak909 hat geschrieben:
22.09.2022 02:52
The firmware has now been updated, and you can update your FujiNet via the flasher to get the fix.
...what I did immediately when I read this - wow that was fast :D

Unfortunately it doesn't fix the problem. When fetching as string I get 2147483647, when fetching as number I get the one of the following numbers depending on the BASIC variant:
  • 2147483647 (Altirra Extended BASIC 0.1)
  • 2147483640 (Altirra 8K BASIC 1.57)
  • 2147483640 (ATARI BASIC C)
I tried XEGS OS and ATARI OS XL 1.3 (with SIO2BT patch). I didn't figure out yet how to use Turbo Basic XL together with FujiNet (my executables of versions 1.4, 1.5, 1.6. 2.0. 2.1, 3.2q crashed at load or when accessing ndev). Here I tried NOS as well as a mounted D1 with Bibo DOS 5.4N with Turbo Basic on it, fnc-tools on D2 and my test program on D3. Loading NDEV.COM works as well as the fnc tools but when accessing N: via Turbo Basic the XIO calls hang. Maybe Turbo Basic corrupts the NDEV handler?

The main problem here seems to be the limits of the math package used by BASIC. Neither the OS nor Altirra BASIC seem to work well with numbers at 10 digits or more.

So in my opinion the best solution would be to parse big numbers as string to avoid these circumstances. (or let the user choose)

Best regards, Daniel

tschak909
Beiträge: 197
Registriert: 17.08.2021 00:22
Has thanked: 4 times
Been thanked: 131 times
Kontaktdaten:

Re: COMING SOON: JSON parser for N:

Beitrag von tschak909 »

Ok, that makes sense. I will have to do some experimentation when I get some cycles. Do you want to experiment with it as well? It's not hard to set up an environment to test small firmware changes.

Instructions on doing your own bring-up are here:
https://github.com/FujiNetWIFI/fujinet- ... rm.IO-code

As for Turbo-BASIC and NDEV, you MUST use @dmsc's relocatable version of Turbo BASIC with NDEV.

-Thom

dabada79
Beiträge: 4
Registriert: 16.02.2022 21:15
Has thanked: 7 times
Been thanked: 1 time
Kontaktdaten:

Re: COMING SOON: JSON parser for N:

Beitrag von dabada79 »

Hi Thom,

I have now set up the development environment and already coded a bugfix which is now in a local branch on my system. On GitHub I added a comment to the (closed) issue #512 so if it would be reopened I could push the branch and initiate a pull request - if this is the workflow you use. Otherwise I need some instructions on how to give you the changes (only the file fnjson.cpp is affected).

BTW: the reason your fix does not work is that you compare the number size with item->valueint which itself is 32 bit only so the check can never be true. Another problem is that the double->stringstream conversion by default rounds the number down to 4 digits so this had to be changed, too. In the end I did a little bit of refactoring the method FNJSON::getValue() so it now accesses the JSON values via getter and not directly by fields. And it logs a bit more (Debug_printf) which made troubleshooting a lot easier :)

tschak909
Beiträge: 197
Registriert: 17.08.2021 00:22
Has thanked: 4 times
Been thanked: 131 times
Kontaktdaten:

Re: COMING SOON: JSON parser for N:

Beitrag von tschak909 »

Thank you so much for fixing this! :)

Hopefully this serves as an example that people can dig into the firmware to fix and add things! :)

-Thom

Antworten

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast