17 replies to this topic

[joeb]

I have a pc set up in my greenhouse that has a controller connected to it via the com port. I use Teraterm to view the data and change it. Teraterm has a feature that will log anything so I can write to a space delimited text file the data coming from this board. In the software on the board I can have this updated every minute, 5 minutes, 10 minutes and up. I basically get temperatures, relays that are open or closed and a date stamp. I could just log into the network from any pc to view this data but want to make a webpage that will give me more information.

I need a way to dynamically update a webpage and read this log file so I can have more information becasue there are 8 temperature sensors and 4 relays I want more details than just 8 temperatures in a row and not remembering what they are and where they are in the greenhouse. I will also use a color background for each temperature so I visually can see green is a go and yellow is a warning I am getting close to red not so good and I have problems.

I would also like to use a text message/email setup when something goes to yellow and red so I can be notified and get out there and do something about it. The web page will be for me to see whats going on anywhere and message system to fix things with a helper. I will be putting 3 more controllers on the PC so once I do the first one it should be easy on the next.

Would PHP be the way to go to do this? I have dabbled with PERL and JAVASCRIPT a long time ago. I have heard about Ajax but do not know anything about it.

[Alexander]

AJAX is not entirely necessary, a simple Javascript refresh every n minutes and a force-reload option may suffice.

The structure of your whitespace delimited file will matter too, are they in rows and columns?

12:22 12 on on on off warning
12:32 15 on on on off warning

You could start by reading the log as an array delimited by lines (whitespace newlines)

$arrayLog = file("/tmp/greenhouse.log", FILE_SKIP_EMPTY_LINES);

You can parse it through various simple loops, however it may depend on the example.

You may wish to look at explode(), file(), arrays and foreach to start with. I believe there are quite a few log parsers out there, I am not sure how to formulate a good general parser in such a little time.

[joeb]

Hi Thanks for the information.

Here is a sample of the text file.

Code:
[Fri Dec 16 20:24:08.343 2011] 000:08:23 0067 75 OPN OPN OPN OPN OPN OPN OPN ON ON ON ON
[Fri Dec 16 20:24:19.937 2011] 000:08:24 0068 75 OPN OPN OPN OPN OPN OPN OPN ON ON ON ON
[Fri Dec 16 20:25:20.843 2011] 000:08:25 0069 75 OPN OPN OPN OPN OPN OPN OPN ON ON ON ON

Will this work like you thought it would?

How did you insert the code in the text box?

[RhetoricalRuvim]

Is there a date-parsing function in PHP like the JavaScript parse () function?

* * *

You might also be interested in the PHP mail () function, which is used for sending emails. I wonder if there is such a thing in JavaScript, though.

* * *

ASP is similar - in a way - to PHP, and you can choose whether to preprocess JavaScript or VBScript, using the '@language' (I think) setting at the top of the file. So yeah, just wanted to add that you can use ASP if you want to use JavaScript, but PHP is fine too.

* * *

In JavaScript, it's String.split (delimiter) and Array.join (glue) functions that do the dividing and adding of strings and arrays.

In PHP, it's the explode (delimiter, string) and the implode (glue, array) functions.

* * *

AJAX is not a separate language, it's a way to use JavaScript to send HTTP requests and get the responses.

* * *

Basically you'd probably have to split the input file contents with the "\r\n" or the "\n" delimiter, depending on how the log file was made, and you'd have an array of single lines that you can scan and process.

Then you can split each line with a space delimiter and look at the different elements. I recommend experimenting with string handling, array processing, playing around, etc., because that helps a lot when making programs. A lot of the times, I also like to test the functions I write before adding them to the big projects that I work on.

[joeb]

I understand that I can use a perl script or javascript to do the parsing of the file but how can I make my website update automatically each time the log file gets a new line written to it? If you look at the example above and I wanted it all in a graph I would use the whole file and import it into excel and make a graph and then publish it to my website maybe every day or week.

But what I would also like is that last line in the example above be the data that is on the webpage. And if I am writing to that text file every minute from my controller I would like the website to get the last line and update it every time. I see how to use a scripting language to parse the file and split it and get the delimiter to divide up the data correctly but can Perl or JavaScript be used to update it on the fly dynamically or is this a function of JavaScript and Ajax or PERL and Ajax?

[RhetoricalRuvim]

I am pretty certain that Facebook chat uses AJAX to wait for new data from the server. Looking at the Firebug console while logged into Facebook, I found that there's usually at least one AJAX HTTP request that seems to be waiting; then I think that request returns and another requests takes over the waiting. But Facebook, from what I've heard, has a lot of resources, that this sort of instant responsiveness is not cheap (post #6 from here).

You could probably also use flash for that, but I don't know much about flash. I just know that you can use ActionScript to make flash programs :D .
Well, I guess I also know very little bit about the structure of flash files, from what I got to reading the flash file format specification; but specifications are not that easy to read, so it would make sense why I'm still not that far into reading that :) .

[Orjan]

I would do this with php, nothing else needed really. Maybe a javascript if you want automatic refresing of the page. The easiest way is to do a page refresh every minute. In worst case, the update was 59 seconds ago, but i think you could live with that delay? If not, refresh more often. Less than 10 seconds between updates is probably not needed. I feel that this is easiest to do with pure php and nothing in javascript.

If you would like lots of historic data treatment, i'll load it into a database.

If you describe the fields after the date a little more, i could give you some better advise, is 75 a temperature and the other OPN the other thermic sensors? What is the 0067...? Just a counter? And what is the time alike thing first after the timestamp?

[RhetoricalRuvim]

Perhaps something like this?:

var http= ... // this should be an XML HTTP request... 

// Use the GET HTTP request method, 

// the script path is changed_yet.php 

// and the query string is filename=log.txt 

http.open ("GET", "/changed_yet.php?filename=log.txt"); 

// When something changes (eg the server accepted the 

// connection, the server responded, etc.), 

// call the check_change () function. 

http.onreadystatechange= check_change; 

// Send the HTTP request to the server. 

http.send (); 


function check_change (){ 

// if the ready state is 4 (if the request is done processing), 

// and if the HTTP status is 200 (no errors), 

// and if the response text (the text that the server sent us back) 

//   contains the word "yes" , then 

if (http.readyState == 4 && http.status == 200 && http.responseText.split ("yes").length > 1){ 

// reload the page. 

location.reload (); 

} else if (http.readyState == 4){ 

// Otherwise, but if the request is done processing, 

// start a new request with the GET method and the same script 

// path and query string; 

http.open ("GET", "/changed_yet.php?filename=log.txt"); 

// when something changes use this function; 

http.onreadystatechange= check_change; 

// send the request. 

http.send (); 

} 

} 

The JavaScript above should keep asking the server if there is new data in the log file (with the filename being log.txt); there is an omitted part in the script (the http= ... part), which I'd leave for you to fill in.

After that, the next step is writing a PHP script file, changed_yet.php, and having it sit there and wait for the file to change, probably occasionally outputting something to let the client know that it's still there. An example of a waiting PHP script could be something like this:

<?php 

// We'll need to free this session so that others can connect to our server while 

// we wait for the file to change, or else we'd get a very unresponsive server, 

// at least for this session, I think. 

session_write_close (); 

$i= 0; 

while ($i < 10){ 

sleep (1); // Sit for one second. 

echo ".";  // Say something, to let the browser know that 

// we didn't just hang up. 

$i++; 

} 

?> 

I think the code above is supposed to wait for 10 seconds, saying "." after each, and exiting. You could probably make something similar but change the waiting time to 60 seconds, maybe, and also making it check to see if the log file has changed, before continuing with the loop. If the file has changed, you could output "yes" and use the break keyword to exit the loop.

At least I think that's how it can work; I haven't tried that stuff yet, but I do have plans for which I have no other ideas than to try similar methods.

[Orjan]

RhetoricalRuvim, why are you doing it so darn complicated? Why all this active processes hanging and draining power of both browser and server, when you could do a easy refresh every x seconds on the browser side? I see no point polling the server all the time, it's not a real time system he's asking about.

[joeb]

Sorry Orjan I had a bunch of stuff that went wrong on my website that I am still trying to fix and did not answer your questions from the post above.

The first one in brackets is like you said the date or timestamp.

The second value is again a timestamp but it is from the start of when you started to log things. So 1 day and so many minutes is the value. I may not use this one.
The third one is a counter like you thought.
The 4th through 11th are temp sensors and if nothing connected are open like you thought as well. I will be putting in wires all around the greenhouse in the next few days to get them to start giving me some data as well as one outside.
The last 4 are the relays. In the setup of the controller you can tell the relay to open when a certain temperature is reached and then as the temperature goes down it will go to off.

The controller has 2 configurations. I have 2 of them right now. I will use one in multi mode where only 1 sensor will control the 4 relays. BUT I can have the other 7 sensors out in the greenhouse just giving me temperatures. That is the one I am asking about to be able to view the data live on a webpage so I can see what is going on. I like the browser side refresh idea you just posted and would like to know more about that idea. I also would like to have it give me a message or text if a temp goes to a low limit in the winter or high limit in the summer that I would like to add to this if possible.

Looking forward to your ideas.

[RhetoricalRuvim]

Orjan said:

RhetoricalRuvim, why are you doing it so darn complicated? Why all this active processes hanging and draining power of both browser and server, when you could do a easy refresh every x seconds on the browser side? I see no point polling the server all the time, it's not a real time system he's asking about.

Because I'm sure that's how Facebook does things with their chat and such.

[Orjan]

I would set up for example xampp on the greenhouse server so it can publish your values. I would setup a database to store the log entries for further usage, as it would be easier to handle in a future. Maybe your sensor system is available to log to an mysql server if available? That would been the best option if so. Please look into that before start partsing the files for your web.

Xampp is a webserver and a database server, open sourced, so it does not cost you anything.
just install it and set it up as services on your machine, and it will automatically start when your computer starts.

RethoricalRuvim, so? Noone have been talking about facebook but you in this thread.