ChipChop Support Forum
Log in
Log out
Join the forum
My Details
REPLIES: 12
VIEWS: 626
BACK
REPLY TO THIS POST
Gizmo
04 May 2023
System Improvement - Web App with WebSockets
In this thread I will be posting updates on this particular work in progress, feel free to reply with suggestions & ideas

Description:

For some time now, I've been wanting to improve one of the functionalities of the ChipChop Web App.
At the moment to get the current status of your devices the Web App is using Ajax and requests a status update from the ChipChop engine on a 10 seconds interval.
That's all fine a peachy because the maximum devices heartbeat interval is 10 sec so at worst you maybe seeing your device status few seconds later than it was broadcast by the device.

But, if you initiate a "trigger event" (they execute instantly) you will not see that reflected in the Web App interface until the next update request cycle happens. For basic use that's absolutely fine, the trigger event will still happen in real-time in the background and it's just a matter seeing it happen.
I mean who needs to stare at the phone waiting to see if a motion sensor has detected motion...right? (yeah, that would be me and I have valid reasons :-)

So...why do I care if the current Ajax method has been working fine for a very long time? Because it can be made better, because Ajax is a http request and they are expensive...and, because I can!

The right solution is to use WebSockets and have a real-time bi-directional communication between the WebApp and the ChipChop engine so trigger-events can be sent instantly to the Web App as they happen.

How difficult is that? Technically, not a big deal, in reality, It's a royal pain in the ass. It needs to keep track in real time who exactly is observing their devices through the WebApp, which devices that belong to that account are on-line and forwarding the trigger events to the exact socket...and all of that without blocking or slowing down anything.

So, I'm rolling up the sleeves and digging into the crap bucket. Let's see how deep this rabbit hole goes :-)


MiniFish92
05 May 2023

Sounds like fun, better you than me, good luck!
Gizmo
15 May 2023

Ok, I've pulled the prototype code I've written last year and tried it first just to remind myself what my logic was.

It seems to work ok but because few things have changed in the main engine it needs some further optimising so no luck with a quick build :-(
Gizmo
20 May 2023

I keep working on/off on this as I have another large project on the go so I keep losing bloody track.

Got the thing working on my development machines but I keep getting weird connect/disconnects when deployed live on the testing server!?

It's really starting to annoy me now, the mechanism work beautifully but not in production mode...wtf?
Jac0b0
20 May 2023

lol it's always like that
what's the problem you have? is it like when you look at it all is fine but when you turn your back it stops working?
Gizmo
21 May 2023

hah yes Jac0b0, exactly that!

That made me think that it's nothing to do with my code but rather this particular server being an asshole, so I've looked into the server logs and I think I know what's going on.


Anyway, long story short, it's all working so just need to do a load test.
Gizmo
10 Jun 2023

Anyone want's to volunteer to help with another load test before I push this feature live?

If you have some spare devices that you don't mind re-routing to the test server for a day or so please let me know and I'll give you details.
Thezzi Mfkr
10 Jun 2023

I can, I have spare ESPs how many do you need?
afadi
10 Jun 2023

Me too, let me know how?
Gizmo
11 Jun 2023

Thanks peeps,

I think 1000 would be good, how many do you have?
Thezzi Mfkr
11 Jun 2023

whtf man, 1000!!!! I mean like 2-3, I don't have a chip factory you know!

you are joking right?
Gizmo
12 Jun 2023

Yep, sorry couldn't resist :-)))))))

2-3 will be great, just need connections from different locations to see how fast it works that's all

I'll send you both details

Anyone else?
Gizmo
03 Jul 2023

Load tests done and everything is working as expected for 2 weeks now so I am closing this project and pushing live the new functionality.

Of course in the process I have discovered another improvement that I want to implement and I will post a new thread when I find the time to dig into it as it's going to be a bigger job.

Many thanks to Thezzi and Aafadi for sacryficing their devices for the good cause (no chips where injured in this process and they are all healthy and recovering well)

Off to the next challenge and to complete the damn tutorials

Cheerio