How to Receive Data from the Cloud (IoT Dashboard)

2019-08-19 | By Maker.io Staff

So far we have created an IoT dashboard that can be viewed over LAN, created a simple API that allows sensors to submit data, and displayed this information graphically… on a graph! In this How-To, we will learn how to further improve our API so that devices can receive data as well as perform other tasks.

BOM

Scheme-It

Scheme-It

Improved API

In the previous How-To articles, we learned how to create an IoT Dashboard that allows devices to submit data. However, our API is far too simple for a real-world application, and some key features that are missing include retrieving data and manipulation of data. Therefore, to rectify this, we will add these additional features as well as create a pair of functions that make interacting with our IoT Dashboard far more simple!

The first change is to the PHP file that will interact with devices. The changes that we will make include the file name, adding command capabilities, and only using one main file for all device operations. Instead of calling our file “submit.php”, we will be calling the file “api.php”. This is a better name because this filename clearly indicates that it is an interface bridge between our devices and the IoT dashboard.

The second major change to our API is that we will integrate command-based response. Before, the device would simply provide a variable name and a data field for saving data, but now devices will provide a third parameter: command. This third parameter tells the API what the connecting device wants to do. The commands that our device can initiate include:

“get” – This will tell the server that we want to retrieve a variable value
“submit” – This will tell the server that we want to submit data to a variable
“clear” – This will tell the server that we want to clear all values in a variable
“delete” – This will tell the server that we want to delete a variable

When our PHP script runs, it first takes each parameter and stores the value into the three main variables (variable, data, and command). A series of ”if” statements then compares the command variable to see what the command is.

Copy Code

<?php
	// Get requested variable from our device
	$variable = $_POST["variable"];
	$data = $_POST["data"];
	$command = $_POST["command"];
	$fileName = 'data/' . $variable . '.txt';
	
	if($command == "get")
	{
		// Get the last entry in the variable file
		
		$fileContents = file($fileName);
		
		$numberOfLines = count($fileContents);
		$lastLine = $fileContents[$numberOfLines - 1];
		
		// Parse the last line to extract the value		
		$strSplit = split(">",$lastLine);
		
		// Send the data back to the device
		echo($strSplit[1]);
	}
	if($command == "submit")
	{
		// Get data the data was sent
		$date = date('m/d/Y h:i:s a', time());
		$dateStr = strval($date);
		
		// APPEND the data to a file
		// TIME : VALUE
		file_put_contents($fileName,strval($date) . '>' . $data  . "\n", FILE_APPEND);	
		
		// Report back the date / time (may be useful in the future for devices with no RTC)
		echo($dateStr);
	}
	if($command == "clear")
	{
		// Replace the contents of the variable file with nothing
		file_put_contents($fileName, "");
		echo("OK");
	}
	if($command == "delete")
	{
		// Delete the variable file
		unlink($fileName);
		echo("OK");
	}	
?>

The “get” command is very simple and starts by loading the variable file into an array called “fileContents”. Since there are many values. we will only return the latest value; this value is always at the bottom of the file, so to get this element, we first start by counting the number of elements in the array “fileContents” and then use this number to get the last element (remembering that arrays are 0-based, so we subtract 1 from the total number of elements). Once the last element has been extracted, we split this line using the “>” character, which separates our value from our timestamp. Then, we simply echo the second string split element, which is our variable value, and our device receives this value as a string.

Copy Code

if($command == "get")
	{
		// Get the last entry in the variable file
		
		$fileContents = file($fileName);
		
		$numberOfLines = count($fileContents);
		$lastLine = $fileContents[$numberOfLines - 1];
		
		// Parse the last line to extract the value		
		$strSplit = split(">",$lastLine);
		
		// Send the data back to the device
		echo($strSplit[1]);
	}

Arduino Code

The API is only half the solution; the other half makes using our API somewhat easier by moving the complicated HTTP code into functions. The two functions that we define are:

void setVariable(String varName, String value)
String getVariable(String varName)

The setVariable function is used to set the value of a variable while the getVariable function is used to retrieve the value of a variable.

String getVariable(String varName)

This function recycles most of the code from the previous articles but there are a few changes. The first change is that the command parameter is now included in the dataToSend variable as seen below:

Copy Code

String dataToSend = "variable=" + varName + "&command=get";

The second change to our code is that the devices continue to stream information in until the server closes the connection. Before, the data reception loop would exit when it detected the terminating series of characters ("\r\n\r\n"), but this occurs before the data has been streamed by the server. Apache servers terminate the connection once they have completed a request, so we take advantage of this by waiting until the server closes the connection.

Copy Code

    while(client.connected())
    {
        memset(dataBuffer, 0x00, sizeof(dataBuffer));
        client.read(dataBuffer, sizeof(dataBuffer));
        receivedData += (const char*)dataBuffer;
    }

To get our returned variable value, we have to start by looking for the special "\r\n\r\n" marker and take note of where this marker is located in our receivedData buffer. To do this, we take advantage of the .indexOf() function, which can tell us the position in a string of a specified string pattern. From here, our data will be located 4 characters after this marker (as the marker is 4 characters long) and the remaining data is our variable! This variable value is extracted using the substring function, which returns the string after the provided character position.

Copy Code

varPtr = receivedData.indexOf("\r\n\r\n") + 4;
return receivedData.substring(varPtr);

Complete Code

Copy Code

// -----------------------------------------------------------------------------------
// Get variable on IoT Platform
String getVariable(String varName)
{
  unsigned int varPtr = 0;
  
  if(client.connect(server, 80))
  {
    // POST Message
    // Create our data packet 
    String dataToSend = "variable=" + varName + "&command=get"; 
    int lengthOfData = dataToSend.length();
    // Send our HTTP data!
    client.println("POST /api.php HTTP/1.0");
    client.println("Host: 192.168.1.103");
    client.println("Content-Type: application/x-www-form-urlencoded");       
    client.println("Content-Length: " + String(lengthOfData));
    client.println();
    client.print(dataToSend);
    
    // Read data from the buffer
    // This data is being ignored in this example
    receivedData = "";
    
    while(client.connected())
    {
        memset(dataBuffer, 0x00, sizeof(dataBuffer));
        client.read(dataBuffer, sizeof(dataBuffer));
        receivedData += (const char*)dataBuffer;
    }
    varPtr = receivedData.indexOf("\r\n\r\n") + 4;
       
    // Stop the current connection
    client.stop();
  }  
 
  return receivedData.substring(varPtr);
}

void setVariable(String varName, String value)

Setting variables is far simpler than getting them, as all we have to do is take the variable value and append it to our HTTP request. The code below shows how this works.

Copy Code

// -----------------------------------------------------------------------------------
// Set variable on IoT Platform
void setVariable(String varName, String value)
{
  if(client.connect(server, 80))
  {
    // POST Message
    // Create our data packet 
    String dataToSend = "variable=" + varName + "&command=submit" + "&data=" + value; 
    int lengthOfData = dataToSend.length();
            
    // Send our HTTP data!
    client.println("POST /api.php HTTP/1.0");
    client.println("Host: 192.168.1.103");
    client.println("Content-Type: application/x-www-form-urlencoded");       
    client.println("Content-Length: " + String(lengthOfData));
    client.println();
    client.print(dataToSend);
    
    // Read data from the buffer
    // This data is being ignored in this example
    receivedData = "";
    
    while(receivedData.indexOf("\r\n\r\n") == -1)
    {
        memset(dataBuffer, 0x00, sizeof(dataBuffer));
        client.read(dataBuffer, sizeof(dataBuffer));
        receivedData += (const char*)dataBuffer;
    }
        
    // Stop the current connection
    client.stop();
  }
}

Complete Example

The code below shows a complete example where our device can submit data and retrieve it using these functions!

Copy Code

#include <WiFi.h>
 
// -----------------------------------------------------------------------------------
// Wi-Fi Variables
const char* ssid      = "TP-Link_B5AC";   // Our Wi-Fi SSID
const char* password  = "93080422";       // Our Wi-Fi Password
 
 
// -----------------------------------------------------------------------------------
// Variables
String variable;
 
// -----------------------------------------------------------------------------------
// TCP Related Variables
WiFiClient client;                        // Create Wi-Fi TCP client
byte server[] = { 192, 168, 1, 103 };     // Our local server
byte dataBuffer[1024];                    // Data buffer to hold data    
String receivedData;                      // Used to hold received data
 
 
// -----------------------------------------------------------------------------------
// Function Prototypes
void setVariable(String varName, String value);
String getVariable(String varName);
 
// -----------------------------------------------------------------------------------
// Main setup function
void setup() 
{
  Serial.begin(9600);
  
  // Connect to Wi-Fi network
  WiFi.begin(ssid, password);
  
  // Wait until connected
  while (WiFi.status() != WL_CONNECTED);
}
 
 
// -----------------------------------------------------------------------------------
// Main Loop
void loop() 
{   
  setVariable("TestVariable", "10");
  variable = getVariable("SolarVoltage");
  Serial.println(variable);
  delay(1000);
}
 
 
// -----------------------------------------------------------------------------------
// Get variable on IoT Platform
String getVariable(String varName)
{
  unsigned int varPtr = 0;
  
  if(client.connect(server, 80))
  {
    // POST Message
    // Create our data packet 
    String dataToSend = "variable=" + varName + "&command=get"; 
    int lengthOfData = dataToSend.length();
            
    // Send our HTTP data!
    client.println("POST /api.php HTTP/1.0");
    client.println("Host: 192.168.1.103");
    client.println("Content-Type: application/x-www-form-urlencoded");       
    client.println("Content-Length: " + String(lengthOfData));
    client.println();
    client.print(dataToSend);
    
    // Read data from the buffer
    // This data is being ignored in this example
    receivedData = "";
    
    while(client.connected())
    {
        memset(dataBuffer, 0x00, sizeof(dataBuffer));
        client.read(dataBuffer, sizeof(dataBuffer));
        receivedData += (const char*)dataBuffer;
    }
    varPtr = receivedData.indexOf("\r\n\r\n") + 4;
       
    // Stop the current connection
    client.stop();
  }  
 
  return receivedData.substring(varPtr);
}
 
 
// -----------------------------------------------------------------------------------
// Set variable on IoT Platform
void setVariable(String varName, String value)
{
  if(client.connect(server, 80))
  {
    // POST Message
    // Create our data packet 
    String dataToSend = "variable=" + varName + "&command=submit" + "&data=" + value; 
    int lengthOfData = dataToSend.length();
            
    // Send our HTTP data!
    client.println("POST /api.php HTTP/1.0");
    client.println("Host: 192.168.1.103");
    client.println("Content-Type: application/x-www-form-urlencoded");       
    client.println("Content-Length: " + String(lengthOfData));
    client.println();
    client.print(dataToSend);
    
    // Read data from the buffer
    // This data is being ignored in this example
    receivedData = "";
    
    while(receivedData.indexOf("\r\n\r\n") == -1)
    {
        memset(dataBuffer, 0x00, sizeof(dataBuffer));
        client.read(dataBuffer, sizeof(dataBuffer));
        receivedData += (const char*)dataBuffer;
    }
        
    // Stop the current connection
    client.stop();
  }
}