Clarification around offline & retry support #1419
Comments
|
Hi @jpalmowski, sorry for the delay. As you have already seen, looking into the code the retry backoff should be kicking in to delay the retry operation (based on your isRetryDisabled: false (which is also the default)). But this check occurs after this one So if the XHR implementation is returning something that satisifes this check that it's likely why the request is getting dropped. LocalStorage: We currently don't have any scheduled work to bring LocalStorage into the App Insights codebase, so I don't have a current timeframe for when this might occur. You are correct though in that because of the general design it's possible, and we do have an internal set of extensions that supports both LocalStorage and IndexedDb but at this point it doesn't work with the App Insights Sender. There is a general plan to move these internal plugins into the public space, but again no committed timeframe. |
|
Hi, @MSNev |
|
Yes, the SDK has a general assumption that it will be online all or most of the time and I don't believe that we have any general plans to assume complete offline or only online sometimes. Currently the SDK (by default) stores events into SessionStorage (controlled by However, this generally doesn't work for Mobile / complete offline scenarios. For those it would need the LocalStorage or IndexedDb support mentioned above as well as some refactoring of the current online / offline listener (currently only used for handling retries) All of this is current managed in the Sender.ts (channel). As part of the SDK, it is built on a "plugin" model (every component is a plugin -- including the Sender.ts), so you "could" implement an OfflinePlugin which when offline simply stored the events into some form of persistent storage and then just not call the "nextPlugin" (thus avoiding the Sender also trying to send the events). And once it detects that it's back online it could simply pickup to persisted events and send them on down the plugin chain there are some internal helpers which you should be able to "recreate" the plugin chain between instances or simply just call track again and when your plugin gets called again you just bypass your logic. If you want to go down this path I can provide some more details. As you mentioned ReactNative, this also seems like something that could be contributed (added) to the ReactNative extension as an optional capability. |
|
Can someone please help me understand this? I have a similar scenario. However, on every refresh I can clearly see that the AI_buffer field in session storage is cleared. Isn't the whole point of session storage that is should persist between refreshes of the same tab? Maybe this is a Ionic/webview thing, but it doesn't seem so. e.g. I can see the session_id in the cookie changing on every refresh. It's a little difficult to test this without Ionic (or some kind of pwa). So I wanted to ask first, is this normal, that the AI-buffer field is cleared on every refresh? If so, why? |
|
The Send buffer should be storing the values in the Session Storage and only clear them out after they have been sent. It also does in fact have an online / offline listener here which (should) be causing the events to be retried. If the events are getting dropped, then your probably hitting the The Sender probably should be changed to not attempt to send requests when offline, and I recall that there is a retry limit (can't find the config ATM). Another option might be to |
|
I'm not really following, honestly. The problem is that the send buffer is almost always empty. For testing purposes I've set the
So, I don't have an issue with DevTools.-.localhost_.2023-03-01.11-14-50-compressed.mp4 |
This can be used like:
```
import { LocalStorageSendBuffer } from "@microsoft/applicationinsights-channel-js";
const appInsights = new ApplicationInsights(...);
appInsights.getSender().setBuffer(LocalStorageSendBuffer);
```
This can be used like:
```
const appInsights = new ApplicationInsights({
config: {
enableSessionStorageBuffer: true,
bufferOverride: "localStorage"
}
});
```
…osoft#1419 This can be used like: ``` const appInsights = new ApplicationInsights({ config: { enableSessionStorageBuffer: true, bufferOverride: { getItem: (logger, key) => localStorage.getItem(key), setItem: (logger, key, value) => localStorage.setItem(key, value), } } }); ```
…osoft#1419 This can be used like: ``` const appInsights = new ApplicationInsights({ config: { enableSessionStorageBuffer: true, bufferOverride: { getItem: (logger, key) => localStorage.getItem(key), setItem: (logger, key, value) => localStorage.setItem(key, value), } } }); ```
…osoft#1419 This can be used like: ``` const appInsights = new ApplicationInsights({ config: { enableSessionStorageBuffer: true, bufferOverride: { getItem: (logger, key) => localStorage.getItem(key), setItem: (logger, key, value) => localStorage.setItem(key, value), } } }); ```
…osoft#1419 This can be used like: ``` const appInsights = new ApplicationInsights({ config: { enableSessionStorageBuffer: true, bufferOverride: { getItem: (logger, key) => localStorage.getItem(key), setItem: (logger, key, value) => localStorage.setItem(key, value), } } }); ```
#2037) This can be used like: ``` const appInsights = new ApplicationInsights({ config: { enableSessionStorageBuffer: true, bufferOverride: { getItem: (logger, key) => localStorage.getItem(key), setItem: (logger, key, value) => localStorage.setItem(key, value), } } }); ``` Co-authored-by: Nev <[email protected]>
#2037) This can be used like: ``` const appInsights = new ApplicationInsights({ config: { enableSessionStorageBuffer: true, bufferOverride: { getItem: (logger, key) => localStorage.getItem(key), setItem: (logger, key, value) => localStorage.setItem(key, value), } } }); ``` Co-authored-by: Nev <[email protected]>
* [BUG] The documentation for enableDebug is incorrect, it should reference enableDebugExceptions #2014 (#2022) * [BUG] SDK LOAD Failure reporting not working #2027 (#2038) * Add a simple interface to enable custom buffer storage solutions #1419 (#2037) This can be used like: ``` const appInsights = new ApplicationInsights({ config: { enableSessionStorageBuffer: true, bufferOverride: { getItem: (logger, key) => localStorage.getItem(key), setItem: (logger, key, value) => localStorage.setItem(key, value), } } }); ``` Co-authored-by: Nev <[email protected]> * [Master] Add readme documentation for IStorageBuffer (#2045) * [Release] Increase version to 2.8.12 (#2046) --------- Co-authored-by: Philip Peitsch <[email protected]>
|
@zolakt a recent patch has been merged (2.8.12+ & 3+) which might allow you to address the issue you're seeing here. By default, AppInsights uses session storage, which is cleared whenever the tab/browser is reloaded, among other conditions. Possibly, it's worth seeing if switching to local storage resolves the issue you're seeing here? Instructions for doing this are outlined on #2037 |
|
FYI - We are now working on a new "extension" which may also help with this. |
|
If you want to see the PR's this (should) identify them |
|
Hi @MSNev :-) Will this latest release (3.1.0), which came out last week, make it possible to support offline situations in a react-native app? |
Depending on your detection of whether your offline -- maybe (not tried this)
No, It's not automatic you will need to also include the new Offline Channel which is used to "store" the batched events into LocalStorage / IndexedDb. There are a couple of additional changes which you will need as well as currently we only have a very basic IOfflineListener implementation (which is only listening to the browser events). We updated the interface to "handle" 2 levels of offline but we don't have any production implementation for this yet. The 2 levels are
@Karlie-777 is leading the development of the Offline Channel and we still need to provide more documentation around how to use it -- And, yes we are still making improvements, so please feel free to provide feedback. |
|
Great work on the 3.1 release! @MSNev @Karlie-777 I'm sure If you'd be willing to nudge us in the right direction with a minimal example how to implement the |
|
Thanks @p-hlp I will update the documentation soon! |
|
@MSNev, @Karlie-777 , could you elaborate more on the below regarding how to implement a custom network detection to override the one from the offlineListener?:
I'm trying to use azure insights in a Mobile App and need to support offline. Using the snippet below peitschie shared in an early comment, to override the StorageBuffer, worked very well for me.
With this I can provide the functions to store the logs on my own IndexedDB to avoid losing it when the user closes the app or is offline. However, as also pointed in the comments above, some times the logs are removed even if not synched with azure. Most of the times it works well, but in some use cases such as when users are using VPNs, the _offlineListener.isOnline() function below still returns true, as it thinks it is connected, and the logs are wiped even if weren't properly synched with azure: If I could somehow implement an override for this function or have this second level of manual checks, I believe it would fix my problem. Do you know if this is possible? Appreciate the help! |
|
@raphaelcr93 so this is the method to get and this is the code to to manually reset offline status |
|
Hey @Karlie-777 , Thank you for your answer! As I'm using my own IndexedDB, I'm using the following snippet only to override the methods of the IStorageBuffer: If it is possible to override or access somehow the current offlineListener without the need to user the plugin, that would be perfect. Another question... |
|
@raphaelcr93 If I understand it correctly, you want to make this to be accessible.I created an issue #2433 here to track the feature request. And if you don't need sdk to manage storing and sending offline events automatically, then offline support plugin is not required. |
|
@Karlie-777 , that's correct, I would like to access the _offlineListener to be able to control the status myself the same way you do in the offline plugin! Do you know if there is any way I could do it manually right now, with the current implementation? |
|
@raphaelcr93 it seems that _offlineListener is not accessible from sender currently, but we will add this feature as soon as possible! |
|
@Karlie-777, Do you have any suggestions to apply as a workaround while we wait for the official release? |
|
@raphaelcr93 I don't believe there is any workaround currently. let me send a quick fix by the end of this Wednesday, and then a preview version which contains the fix should be released no later than Thursday. |
|
@raphaelcr93 preview version is published https://www.npmjs.com/package/@microsoft/applicationinsights-channel-js/v/3.3.4-nightly3.2410-03 |
|
Thank you @Karlie-777 , I will test it and let you know if there I have any issues with that. |
|
Hey @Karlie-777, I'm having some issues building the module. Any chance you have a CDN version with this fix just like the current one for 3.3.3? |
|
@Karlie-777, can you point out how to access the offlineChannel from Javascript in this new fix? |
|
@Karlie-777, actually found it! For future reference if anyone need it, here is how I got it: And then you can set the state with: offlineListener.setOnlineState |
|
@Karlie-777, the feature worked! Just a note, though! I also noticed, that there were some rules around AdBlock to not check for status equal to 0: To workaround it in my use case, I had to apply a slightly change to the script in this line: What I did is to always check if the state was offline, and in this case always go inside the if to check if it needs retry: Not sure if it makes sense to add in the original code, but if yes, that's what I did and is working very well so far. |
|
Use this #2436 to Track changes |
|
Hey @Karlie-777, Just found another use case that you might be able to help me with. I noticed that when we initialize the AppInsights again the unsent data that was stored IStorageBuffer (Defined using the bufferOverride property) is completely wiped. In this case what happens is that if the user closes the page and open it again the data is overriden by a a new set of events, even if the previous data was still stored in the indexedDB, . I had a look at the properties 'enableSessionStorageBuffer' and 'disableDataLossAnalysis' from the main page, hoping it would help, but no luck. Do you know about any way to avoid losing the data that was still on the buffer after initializing the appInsights library? |
|
@raphaelcr93 when your own bufferOverride is provided, all setItem and getItem will use the overridden ones. enbableSessionStorage is mainly used when bufferoverride is not provided. |
|
Hey @Karlie-777 , sorry for taking so long to answer! I fixed it by not using promises, but I still wasn't able to make it work after refreshing a page or closing the app and reopening it. I noticed that every time we hit the refresh button, for some reason the setItem override action is called and wipe the data.
However, the data is not sent to the server. And as expected when the page loads again all buffers are empty so no data is retried. At this point I'm considering to try your plugin if it was tested and is working. Thank you in advance! |
|
@raphaelcr93 the set and get function is designed to be sync. Since indexed DB itself is async, saving/getting data from indexed DB might not be in the expected order. And If you have your own sync storage, it should be similar to session storage otherwise parsing events function might not work. And I don't believe we have offline support deployed to CDN |
|
Hi @Karlie-777 , Yes I'm using them synchronously. However, before refreshing the screen or closing it, the override actions are still called cleaning the buffers. Here, I made a quick sample. Used this to initialize app Inisights: As a result, when I click refresh, the setItem override actions are called and the data is wiped without sending to insights:
Is this how it is supposed to work? EDIT:
|
|
are there any page unload events triggered during the process? |
|
Hey @Karlie-777, That was it! Thank you for pointing to the right direction. |
Hi I'm just seeking some clarification around the behaviour I'm seeing when the browser is offline
I can see code for exponential back off on retries so I'm assuming this should be a supported scenario but wanted to verify before I debugged any further.
I'm also seeking any information on if/when localStorage is going to be supported. It seems like it used to exist (#602) and I can see some lower level util code to support it - Given your architecture is pluggable and the interfaces for session and local storage are the same I'm wondering why this hasn't been included?
My config if it helps:
The text was updated successfully, but these errors were encountered: