ESP8266 crashes when is receiving data from websocketserver

[elC0mpa]

Describe the bug
I am using your library to receive a JSON file from a websocketserver. When it receive the file, it starts printing hexadecimal numbers and after two seconds aproximately the esp8266 restarts.

To Reproduce
Version : 0.4.11
Board : NODEMCU
Only using this library

Expected behavior
I expect to receive the json file, wich is received as a string, and later i want to print it By using serial library

Additional context
I supposed that this problem was related to the receive buffer size, so I execute the same code, but receiving a short json file and it worked as expected, so I think the problem is not related to code.

[gilmaimon]

Hi Jose,

Sounds weird, let's try to solve it. Can you write what software you use on the server? I'll try to setup a similar server and see if I have issues with long messages as well. If I can access the same server you are using it will be best (if it's a public service).

Also, can you turn on the esp8266 debug logs (from the boards menu) and attach those.

Thanks,
Gil.

[elC0mpa]

Thanks for your answer @gilmaimon, I can not publish the server code because i don't have it and it is written in php. I need to comunicate with that server wich will be running on a Raspberry pi. Let me ask you something, which is the maxium lenght of received data in this library, because i am pretty sure that this is the problem. Remember it worked with the same code when i changed the size of the json file.

[gilmaimon]

Hi Jose,

There is no limitation.

How big is the file you are getting, could the esp be out of memory? Do you have an esp32 around?

[adelin-mcbsoft]

Hi Jose,
Let me say few words, perhaps will help you:

1. First of all, you told me you are using PHP backend, which makes me think you are using some websocket implementation found on the internet;
Most of those I found in the beginning, do not really read the headers properly and cannot handle long messages, they just truncate it or throw an error.
Been there, done that, ended up creating my own correct implementation by reading the RFCs, works like a charm now, HOWEVER

2. I did some testing on few Arduino modules, though I work with ESP32. The maximum amount I could safely send was 16kb, working with it as a client. I sometimes reached more (about 24kb), but from what I've tested, this was the safest amount of data to send. Anyway, should be a lot. I'm talking here about one single message, you could send as much as you want in multiple messages, by encoding & chunking it.

I don't think its a limitation of the library (and @gilmaimon confirmed so), but rather of the ESP32's Arduino libraries that handle the strings. Didn't investigate further, as 16kb were enough for me. Even the browsers limit the amount of data you can send/receive (and that's 128kb); seems a bit more, but browsers have much more memory than an micro controller.

Hope it helps a bit,
All the best,
A.

[elC0mpa]

Thanks for your answer @gilmaimon and @adelin-mcbsoft. First of all, I have no an Esp32 to test if the problem is related to RAM, but i am pretty sure that the file is not so big. I will check it later.

1. First of all, you told me you are using PHP backend, which makes me think you are using some websocket implementation found on the internet;
Most of those I found in the beginning, do not really read the headers properly and cannot handle long messages, they just truncate it or throw an error.
How can i know if this php implementation is correct? Remember that using the same code, I changed only the json file and everything worked as expected. Maybe it is something related to frames, but i am not sure.
Thanks for your answer

[gilmaimon]

Cool, let us know when you know more about the file size.

Regarding the correctness of the implementation. I assume it is correct, but might require the use of some extension I'm not supporting. Anyway, it shouldn't crash the esp.

If you can use EspExceptionDecoder to decode the crash stacktrace (those hex bytes) it will help identify what exactly is failing.

Also I'll point out again those ESP8266 debug logs, which might help.

Thanks,
Gil.

[elC0mpa]

Ok @gilmaimon, let me tell you something. First of all, I used the ESPAsyncWebServer to simulate the same behavior as the php websocketserver. By doing this i can identify if the problem is related to the server or to the client. Because of the problem remains, I was able to detect that the problem is not related to the server. The file size is about 4 kB, reason why we can descart that the esp is running out of memory.
I modified the server code to send Just a part of the file, not entirely. Because i send the json file as a string, I just decided to send 500 chars, and it worked. But after that i decided to send 600 chars and the Esp01 which is acting as the websocketclient crashed. Tomorrow i will take a look to EspExceptionDecoder, I hope it could Help me

[gilmaimon]

Hi Jose,

I don't think this is a library limitation or bug. There is no difference between message of size 500 and of size 600 (considering the code which runs inside the library in each case). Also, the same logic running on my PC can handle 4K messages with no problem - which is also why I don't think this is a logic bug.

Best thing to do is to decode the stack trace, this will let us know if the crash is related to memory or something else. I wouldn't discard running out of memory, the memory usage can always be improved and there are other stuff using your esp RAM. Also, some of the memory is used for the internal TCP buffers (I think around 8K?).

Best steps to take are:

• Share the code you are using exactly (if you don't mind sharing)
• Decode the crash and print the decoded stack trace (make sure to censor private information like paths on your PC)

I'm sure we will sort this out.
Gil.

[elC0mpa]

Thanks @gilmaimon, I will follow your advice and later i will tell you how it was. Thanks for your answer

[elC0mpa]

Hello @gilmaimon, I already used the espExceptionDecoder, and it wasn't succesfull. The thing is, that it looks like my problem is not an exception. When it happens, the serial monitor shows : soft WDT reset and after that, it shows the stack trace, so it doesn't decodify any exception. I will send you the json file, and maybe you could Help me By receiving it using your library. I will be waiting for your answer, thanks in advance

[gilmaimon]

Jose, look how deep we are into this discussion and this is the first time you mention that the exception is a soft WDT reset...

1. Please copy and paste the crash log here
2. Please share the code you are using
3. Please try again to decode the stack trace
4. The error you mention is called because esp8266 has a thing called software watchdog. The watchdog will reset your sketch if it is stuck for too long. It is possible that receiving a 4k file just takes too long. It is also possible that you have some other code that is causing the issue.

Helpful resources:

• https://arduino-esp8266.readthedocs.io/en/latest/faq/a02-my-esp-crashes.html
• https://forum.arduino.cc/index.php?topic=442570.0

Gil.

[elC0mpa]

Thanks for sharing these links @gilmaimon, I learned some things By reading them, I decided to share my code and my json file with you. I really hope you could Help me. I followed the steps of how to use the espExceptionDecoder, but it doesn't worked. Despiste this, I am pretty sure that the soft WDT overflows in the line number 22, which is the line where i read the incoming msg from the websocketserver. Thanks in advance
Json file

{
   "clientes": {
     "1": {
       "id": 1,
       "nombre": "Cliente 1",
       "edad": 29,
       "sexo": "M",
       "preferencia_tv": "Deportes",
       "color": "#fff",
       "temperatura_agua": "fria",
       "mascota": 1,
       "carro": 0,
       "created_at": "2019-09-24 19:58:55",
       "updated_at": "2019-09-24 19:58:55",
       "tiempo_en_area_simulacion": 4,
       "tiempo_en_area": 4,
       "tiempo_restante_en_area": 0,
       "area": {
         "id": 6,
         "nombre": "Piscina",
         "icono": "fa-support",
         "tiempo_en_area": 4,
         "created_at": "2019-09-24 19:58:55",
         "updated_at": "2019-09-25 15:06:35"
       }
     },
     "2": {
       "id": 2,
       "nombre": "Cliente 2",
       "edad": 29,
       "sexo": "M",
       "preferencia_tv": "Deportes",
       "color": "#fff",
       "temperatura_agua": "fria",
       "mascota": 1,
       "carro": 0,
       "created_at": "2019-09-24 19:58:55",
       "updated_at": "2019-09-24 19:58:55",
       "tiempo_en_area_simulacion": 2,
       "tiempo_en_area": 4,
       "tiempo_restante_en_area": 2,
       "area": {
         "id": 6,
         "nombre": "Piscina",
         "icono": "fa-support",
         "tiempo_en_area": 4,
         "created_at": "2019-09-24 19:58:55",
         "updated_at": "2019-09-25 15:06:35"
       }
     },
     "3": {
       "id": 3,
       "nombre": "Cliente 3",
       "edad": 29,
       "sexo": "M",
       "preferencia_tv": "Deportes",
       "color": "#fff",
       "temperatura_agua": "fria",
       "mascota": 1,
       "carro": 0,
       "created_at": "2019-09-24 19:58:55",
       "updated_at": "2019-09-24 19:58:55",
       "tiempo_en_area_simulacion": 1,
       "tiempo_en_area": 1,
       "tiempo_restante_en_area": 0,
       "area": {
         "id": 1,
         "nombre": "Habitaci?n Ba?o",
         "icono": "fa-bath",
         "tiempo_en_area": 1,
         "created_at": "2019-09-24 19:58:55",
         "updated_at": "2019-09-24 19:58:55"
       }
     },
     "4": {
       "id": 4,
       "nombre": "Cliente 4",
       "edad": 29,
       "sexo": "M",
       "preferencia_tv": "Deportes",
       "color": "#fff",
       "temperatura_agua": "fria",
       "mascota": 1,
       "carro": 0,
       "created_at": "2019-09-24 19:58:55",
       "updated_at": "2019-09-24 19:58:55",
       "tiempo_en_area_simulacion": 1,
       "tiempo_en_area": 1,
       "tiempo_restante_en_area": 0,
       "area": {
         "id": 4,
         "nombre": "Parqueo - Mascota",
         "icono": "fa-github-alt",
         "tiempo_en_area": 1,
         "created_at": "2019-09-24 19:58:55",
         "updated_at": "2019-09-24 19:58:55"
       }
     }
   },
   "preferencias": {
     "id": 1,
     "limite_de_basura": 100,
     "coeficiente_de_basura": 2,
     "tiempo_de_simulacion": 1,
     "dia": "07:00 AM",
     "noche": "07:00 PM",
     "cant_actuadores_energia": 5,
     "cant_actuadores_agua": 5,
     "created_at": "2019-09-24 19:58:55",
     "updated_at": "2019-10-02 16:07:54"
   },
   "simulacion": {
     "basura_acumulada": 100,
     "dia": 176,
     "hora": 18,
     "cantidad_clientes": 4,
     "sesion": "dia"
   }
}

platformio code

#include <Arduino.h>
#include <ArduinoWebsockets.h>

#define SSID "AsyncWebSocketTest"
#define PSWD ""
#define PORT 82
#define IPADDRESS "192.168.0.1"
#define ROUTE "/"
#define TEST true
#define BAUDRATE 115200

#define USE_SERIAL Serial

using namespace websockets;

WebsocketsClient client;

#pragma region  Events

void onMessageCallback(WebsocketsMessage msg)
{
  auto data = msg.data();
  Serial.println(data);
}

void onEventsCallback(WebsocketsEvent event, String data)
{
  if (event == WebsocketsEvent::ConnectionOpened)
  {
    Serial.println("Connection Opened");
  }
  else if (event == WebsocketsEvent::ConnectionClosed)
  {
    Serial.print("Connection Closed: ");
    auto code = client.getCloseReason();
    Serial.println(code);
  }
  else if (event == WebsocketsEvent::GotPing)
  {
    Serial.println("Got a ping");
  }
  else if (event == WebsocketsEvent::GotPong)
  {
    Serial.println("Got a pong");
  }
}
#pragma endregion

void setup() {
  // put your setup code here, to run once:
  USE_SERIAL.begin(BAUDRATE);

	USE_SERIAL.setDebugOutput(true);

	USE_SERIAL.println();
	USE_SERIAL.println();
	USE_SERIAL.println();

	for(uint8_t t = 4; t > 0; t--) {
		USE_SERIAL.printf("[SETUP] BOOT WAIT %d...\n", t);
		USE_SERIAL.flush();
		delay(1000);
	}

  WiFi.mode(WIFI_STA);
	Serial.println("Trying to connect...");
	WiFi.begin(SSID, PSWD);
	Serial.println("Connected to Wifi");
	delay(1000);
	Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
	  
	client.onMessage(onMessageCallback);
  client.onEvent(onEventsCallback);
  
  #if TEST
  client.connect(IPADDRESS, PORT, ROUTE);
  #else
  client.connect("ws://192.168.1.11:6001");
  #endif
  client.ping();
}

void loop() {
  // put your main code here, to run repeatedly:
  client.poll();
}

EDIT: I've allowed myself to edit your comment just so the code is easier to read (@gilmaimon)

[gilmaimon]

I'm having some troubles with my esp8266, but I think I solved it.
Can you try again with the changes commited to master?

Gil.