Add CHARTS to Meters

May be in a first time (and it would be nice to have it anyway), it could be nice to be able to export events into csv in order to use "the power of excel" (or open office obviously)

http://community.zipato.com/responses/add-charts-to-meters

or simply create a google spreadsheet device/endpoint (from formkey) that saves values via a Form

1. https://docs.google.com/spreadsheet/embeddedform?formkey=xxxxxxxxxxxxxxxxxxxxxxxx&entry.0.single=value

Thanks, in the meanwhile i use sen.se to graph it, but it would make sense to have it in zipato gui, domotics also means metrology for me.

I asked for a login on sen.se, thank you sebastien, it looks interesting.

Did not make the form+spreadsheet myself but should be quite direct with a HTTP request. The problem is that today the zipabox can't get parameter values (eg temperature) and put it in a HTTP request (from what I saw - it's another idea here). So I think it can't work right now ;-) Hope it will be possible soon !

@Sebastien also requested a sen.se access yesterday. How do you send the parameter values to the sen.se platform ?

here is a post explaining Cheap and easy event hooks using Google Spreadsheet Forms [Event Tracking]

Thank you, very interesting.

I did the post about the parameters ;)

in fact my data are not send to sen.se from my zipato box, but from a soft i write that get data from my oregon devices and send them both to zipato and sen.se.

This also mean that unfortunately i can't graph my zwave sensors, but in my case most of my zwave are 'actionners' not sensors.

Any news ?

Another site that can be used to do charts externally: https://www.thingspeak.com

But to use it we need to add parameters in a http-request like with google forms

We will publish this feature very soon.

I made a video about Thingspeak (even if at that time it is not possible to use parameters), it is in french, but the sound is not really mandatory:

http://www.youtube.com/watch?v=eU2bpfeGNrg

• I am also very interested by this feature

could you tell us what means very soon please ? are talking in days / weeks / months ?

Tks,

we need to publish few new features and expansion modules first. You can expect this feature to be released before July.

ok tks for clarification

Hi, meanwhile I would like to be able to gather meters values within tools like sense.se thingspeak or Cosm, I had figure out how to use the rest API to do it, but I'm only missing the format of the password parameter for the login method. could you please tell me how to generate it, I guess it has to be a SHA1 hash of plain text password and nonce, but I don't know the exact format.

Help would be appreciated

Gregory: Without the HTTP request with variable we are kinda stuck inside the Zipato dashboard

Not really in fact, thanks to the rest API, you can poll values from your Zipabox and use it in other applications. I finnaly manage to reverse engineer the API calls need for that purpose. My script is only a proof of concept, it poll every 5 minutes the value of a temperature sensor and publish it to sen.se. Next step is to have it done on my raspberry and polish the code.

I would love to know how you did :)

do you talk about the rest API of the box or the cloud service one ?

Gregory...please share

I'll prepare a thread describing the process. Hope to be able to post it tomorrow

Thanks in advance Gregory !

Thanks Gregory!!

Hi,

Sorry but I have to too much time to spend on writing this thread so it gives only the baseline, I'll try to write a more polished article later with code example.

So in a nutshell, to get the meters values, you have to deal with the REST Api (same used by the Iphone application)

1) do a GET request to URL "http://my.zipato.com:8080/zipato-web/json/Initialize", this will return in teh response header a JSESSIONID and a NONCE values. all your next request will have to get the JSESSIONID in the headers, NONCE is a randomly generated number for unique use only (see point 2).

2) Then do a POST request to URL "http://my.zipato.com:8080/zipato-web/json/Initialize" with parameters 'username','password' and 'method' in the post data, username value is your login, method has to be set to 'SHA1', and password has to be generated as following : SHA1(nonce + SHA1(plain_text_password))

Now that you're authenticated you can start using the REST API.

-> url "http://my.zipato.com:8080/zipato-web/rest/meters/" returns full information about all meters

-> url "http://my.zipato.com:8080/zipato-web/rest/meters/values/" returns only values of meters. Meters are indentified by GUID, You have to analyze return of both urls mention above in order to map those GUID with your meters in the my.zipato dashboard.

Have fun, Grégory

tks a lot for that. could you share you script as it is so we can adapt it to our needs ?

Basically I tried to do what you said above in php and it doesn't work. When I do the post step 2, it gives back same as in step one and when I ask for meter I get NULL.

Tks,

Ok I get it work. For those who are insterested please find below my php code very dirty but I'm sure you'll clean it yourself :)

1. <?php
2. $url = 'http://my.zipato.com:8080/zipato-web/json/Initialize';
3. $username = XXXX;
4. $password = XXXX;
5. $json = json_decode(file_get_contents($url));
6. $sid = 'jsessionid';
7. $none = 'nonce';
8. $id = $json->{$sid};
9. $nonce = $json->{$none};
10. $temp = sha1 ($password);
11. $temp2 = $nonce.$temp;
12. $pass = sha1($temp2);
13. $data = array("username" => "$username", "password" => "$pass", "method" => "SHA1" );
14. $data_url = "username=$username&password=$pass&method=SHA1";
17. $ch = curl_init('http://my.zipato.com:8080/zipato-web/json/Login');
18. curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
19. curl_setopt($ch, CURLOPT_COOKIE, 'JSESSIONID='.$id);
20. curl_setopt($ch, CURLOPT_POSTFIELDS, $data_url);
21. curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
23. $output = curl_exec($ch);
25. $c = curl_init();
26. curl_setopt($c, CURLOPT_URL, "http://my.zipato.com:8080/zipato-web/rest/meters/values/");
27. curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
28. curl_setopt($c, CURLOPT_COOKIE, 'JSESSIONID='.$id.'; Path=/zipato-web/; HttpOnly');
29. curl_setopt($c, CURLOPT_HEADER, false);
30. $output = curl_exec($c);
31. var_dump($output);
32. ?>

Xmeslin: where do implement this php code? How do you fetch the data to sen.se?

Thanks for explaining for a rookie

I have a linux server (NAS) doing that.

This php code is called every 5 min with cron.

I modified a bit yesterday and now I store all my temps in a mysql database and graph it with highstock. If you want to push to sen.se you' would need to change my sql query to http.

See below : Again pretty dirt but I wanted to start graph very fast and I'm not a dev guru

1. $c = curl_init();
2. curl_setopt($c, CURLOPT_URL, "http://my.zipato.com:8080/zipato-web/rest/meters/values/");
3. curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
4. curl_setopt($c, CURLOPT_COOKIE, 'JSESSIONID='.$id.'; Path=/zipato-web/; HttpOnly');
5. curl_setopt($c, CURLOPT_HEADER, false);
6. $output = curl_exec($c);
7. $c = curl_init();
8. curl_setopt($c, CURLOPT_URL, "http://my.zipato.com:8080/zipato-web/rest/meters/");
9. curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
10. curl_setopt($c, CURLOPT_COOKIE, 'JSESSIONID='.$id.'; Path=/zipato-web/; HttpOnly');
11. curl_setopt($c, CURLOPT_HEADER, false);
12. $output = curl_exec($c);
13. $json = json_decode ($output,true);
14. $con = mysql_connect($mysql_host,$mysql_user,$mysql_pass,$mysql_port);
15. if (!$con) {
16. die('Could not connect: ' . mysql_error());
17. }
18. mysql_select_db("teleinfo", $con);
19. $timestamp = time();
20. $date = $today = date("Y-m-d H:i:s");
21. foreach ($json as $meter) {
22. //var_dump($meter);
23. $name = $meter['name'];
24. foreach ($meter['attributes'] as $attribute)
25. {
26. if ($attribute['definition']['name'] == "TEMPERATURE")
27. {
28. $temp = round($attribute['value'],1);
29. mysql_query("INSERT INTO temperature (timestamp,date,piece,temperature) VALUES ('$timestamp','$date','$name','$temp')");
31. }
32. }
33. }
34. mysql_close($con);
35. ?>

Ok I did a try for sen.se and please find a code example:

1. $url = "http://api.sen.se/events/?sense_key=xxxxxxxxxx";
2. $timestamp = time();
3. $date = $today = date("Y-m-d H:i:s");
4. foreach ($json as $meter) {
5. //var_dump($meter);
6. $name = $meter['name'];
7. echo "$name\n";
8. foreach ($meter['attributes'] as $attribute)
9. {
10. if ($attribute['definition']['name'] == "TEMPERATURE")
11. {
12. $temp = round($attribute['value'],1);
13. if ($name == "Temp Julie")
14. {
15. echo "Ok";
16. $data = array('feed_id' => '12345','value'=> $temp);
17. $content = json_encode($data);
18. $curl = curl_init($url);
19. curl_setopt($curl, CURLOPT_HEADER, false);
20. curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
21. curl_setopt($curl, CURLOPT_HTTPHEADER,array("Content-type: application/json"));
22. curl_setopt($curl, CURLOPT_POST, true);
23. curl_setopt($curl, CURLOPT_POSTFIELDS, $content);
24. $json_response = curl_exec($curl);
25. var_dump($json_response);
26. }
27. }
28. }
29. }

i can give example in Python if needed !

Hello,

Have you new of this function

thank you

Could someone please explain us how to extract and record data from meters please? ;)

As far as I am concerned, I own a nas (synology) and I know that if I host a php file on it, I can use it to call functions or send commands...etc (I use one for turning on/off wifi with a zipabox rule).

But could anybody explain exactly how to record data directly on the nas and display a graphic?

and can run add a entry in cron ?

Do you have some basics on scripting / programming ? If yes I'll tell how use php + mysql + highchart to do it

Well actually it seems pretty easy now to execute a script or command with DSM 4.3 (synology os)

I found a tutorial which explain how to do it.

If you can post the files (with XXXX where I need to replace data) and tell me where to put them/how to name them, maybe I could find by myself how to use them.

Btw: si tu préfère m'expliquer en français c'est tout à fait possible ;)

ok phase 1 : run the script below every 10 min. It will take all temperature and put them in a database. you need to create it first with the field and it also send 1 temp (Julie) to an online tools for example sen.se :

1. <?php
3. include ("config.php");
5. $url = 'http://my.zipato.com:8080/zipato-web/json/Initialize';
6. $username = 'XXXXXXX';
7. $password = 'XXXXXXX';
9. $json = json_decode(file_get_contents($url));
10. $sid = 'jsessionid';
11. $none = 'nonce';
12. $id = $json->{$sid};
13. $nonce = $json->{$none};
17. $temp = sha1 ($password);
18. $temp2 = $nonce.$temp;
19. $pass = sha1($temp2);
21. $data = array("username" => "$username", "password" => "$pass", "method" => "SHA1" );
22. $data_url = "username=$username&password=$pass&method=SHA1";
26. $ch = curl_init('http://my.zipato.com:8080/zipato-web/json/Login');
27. curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
28. curl_setopt($ch, CURLOPT_COOKIE, 'JSESSIONID='.$id);
29. curl_setopt($ch, CURLOPT_POSTFIELDS, $data_url);
30. curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
33. $output = curl_exec($ch);
36. $c = curl_init();
37. curl_setopt($c, CURLOPT_URL, "http://my.zipato.com:8080/zipato-web/rest/meters/values/");
38. curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
39. curl_setopt($c, CURLOPT_COOKIE, 'JSESSIONID='.$id.'; Path=/zipato-web/; HttpOnly');
40. curl_setopt($c, CURLOPT_HEADER, false);
42. $output = curl_exec($c);
46. $c = curl_init();
47. curl_setopt($c, CURLOPT_URL, "http://my.zipato.com:8080/zipato-web/rest/meters/");
48. curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
49. curl_setopt($c, CURLOPT_COOKIE, 'JSESSIONID='.$id.'; Path=/zipato-web/; HttpOnly');
50. curl_setopt($c, CURLOPT_HEADER, false);
52. $output = curl_exec($c);
53. $json = json_decode ($output,true);
57. $con = mysql_connect($mysql_host,$mysql_user,$mysql_pass,$mysql_port);
59. if (!$con) {
60. die('Could not connect: ' . mysql_error());
61. }
63. mysql_select_db("teleinfo", $con);
65. $url = "http://api.sen.se/events/?sense_key=ZZZZZZZZZZZZZZZZZ";
67. $timestamp = time();
68. $date = $today = date("Y-m-d H:i:s");
69. foreach ($json as $meter) {
70. //var_dump($meter);
71. $name = $meter['name'];
72. foreach ($meter['attributes'] as $attribute)
73. {
74. if ($attribute['definition']['name'] == "TEMPERATURE")
75. {
76. $temp = round($attribute['value'],1);
77. mysql_query("INSERT INTO temperature (timestamp,date,piece,temperature) VALUES ('$timestamp','$date','$name','$temp')");
78. if ($name == "Temp Julie")
79. {
80. $data = array('feed_id' => 'YYYYYY','value'=> $temp);
81. $content = json_encode($data);
83. $curl = curl_init($url);
84. curl_setopt($curl, CURLOPT_HEADER, false);
85. curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
86. curl_setopt($curl, CURLOPT_HTTPHEADER,array("Content-type: application/json"));
87. curl_setopt($curl, CURLOPT_POST, true);
88. curl_setopt($curl, CURLOPT_POSTFIELDS, $content);
90. $json_response = curl_exec($curl);
91. }
92. }
94. }
95. }
97. mysql_close($con);
99. ?>

config.php contains only info to connect the mysql database:

1. <?php
2. $mysql_host = "W.W.W.W";
3. $mysql_user = "WWWW";
4. $mysql_pass = "PPPPP";
5. $mysql_port = "3306";
6. ?>

once you have your database, you can easily chart it using library such as highstock hichchart. My code at home is pretty complex so I'll need to simplify it before posting it.

Let me know if you need help here too.

Wow... ;)

Thanks for sharing!!

It seems harder to understand than what I expected ^^I ll have to give it a try but I think it will be a little bit difficult for me...I already configured the files with my own sen.se key and user/pass for the login of zipabox/mysql but I m not sure about how I can extract data in the mysql db

basically there are 2 options.

Either you use cloud chart such as sen.se and then you do not need to store in mysql database but you rely on cloud/internet connection or you want to be able to work offline and then you need mysql + chart site hosted on the NAS such as high chart.

Hello,

I just saw in the general settings of the box that there is a paid service (monthly fee) to have charts !!!!

Is it a joke or we will have to pay a monthly fee to get one of the most basic feature ?

Hope this will change ...

In all case there is no way to buy it...

Look here:

http://community.zipato.com/responses/new-parameters-wallet-and-services-_3

Here is an update regarding my configuration.

I bought a Raspberry Pi and a Aeon Labs Z-Stick (Zwave USB interface).

There are multiple free software solutions to make them work.

I finally chose agocontrol, which relies on a AMQP bus to manage zwave messages (and the openzwave library). It's a very open solution and it can be extended with python clients.

I attached 3 everspring ST814 multilevel sensors, which were originally attached to my zipabox. My goal was to send temperature and humidity values to the sen.se platform. Agocontrol comes with a working datalogger to a sqlite db in standard.It took me a few days to fully understand the agocontrol way of programming zwave events handlers. It's now done and it works very well. One of the benefits of this approach is that it's a full "push" approach (without any polling). When the sensor updates info (depending on how it's configured) the event handler simply pushes the same info to sen.se, which notably provides analysis and graph capabilities out of the box. The only "intelligence" to implement is the mapping between zwave devices and sen.se feeds (done it with a json config file).

Will see how it behaves in the long run. But it seems to be the best approach regarding configure once / run continuously devices.

Is there any project starting to have something similar to the myVera project? I guess with the new API and scripts like xmeslin above, it would be possible...

I do not know the project but : API + mysql + highcharts library => chart very easy to do and customize to look exactly as you want.

@xmeslin: I wish it would be that easy for me too ^^

Maybe one day someone will create a detailed tutorial to help people to set this up step by step ;)

I already code a python script with a configuration file to easily set-up graphs on sens.se. I need to test it a little bit and then to build a doc. It is not as good as the graphs from zipato or an app with highchart library but it can be a good start! From this script it would be easy to improve it and store data in database and than to add a module to graph with highcharts

Cool! I wish it will help people like me ;)

Here are two snapshots of what I configured on my sen.se dashboard.

works with agocontrol on Raspberry Pi and a single python event handler which sends data to sen.se and also to virtual devices on my zipabox.

Have been running fine for a few weeks (with a few reboots although).

Won't have time to write a doc in the next few weeks.

Contact me if you're interested in the python script. Basically the main part looks like this:

2. # sends data to sense
4. def sendToSenSe(self, deviceid, level):
5. feed_id = self.getSenseFeedId( deviceid)
6. if feed_id != None:
7. data = json.dumps({"feed_id": feed_id, "value": level})
8. try:
9. urllib.urlopen('http://api.sen.se/events/?sense_key='+self.sense_key, data)
10. except:
11. syslog.syslog(syslog.LOG_ERR, "Cannot open http connection to sen.se !")
12. pass
14. # sends data to zipabox
16. def sendToZipabox(self, deviceid, level):
17. url = self.getZipaboxURL( deviceid)
18. if url != None:
19. try:
20. urllib.urlopen(url+str(level))
21. except:
22. syslog.syslog(syslog.LOG_ERR, "Cannot open http connection to zipato !")
23. pass
26. def eventHandlerMeter(subject, content):
28. # get info from message content
29. if 'uuid' in content:
30. uuid = content["uuid"]
31. else:
32. uuid = ""
33. if 'unit' in content:
34. unit = content["unit"]
35. else:
36. unit = ""
37. if 'level' in content:
38. # convert degF to degC and round to xx.x format
39. if unit == "F":
40. level = round( (float(content["level"])-32)/1.8, 1)
41. unit = "C"
42. else:
43. level = round( float(content["level"]), 1)
44. else:
45. level = 0.0
47. #sending message to sense & Zipabox
48. if uuid != "":
49. deviceid = client.uuidToInternalId( uuid)
50. syslog.syslog(syslog.LOG_NOTICE, "Processing Sen.se/Zipabox Event: uuid: %s deviceid: %s unit: %s, level: %f" % (uuid,deviceid,unit,level))
51. mapping.sendToSenSe( deviceid, level)
52. mapping.sendToZipabox( deviceid, level)
53. else:
54. syslog.syslog(syslog.LOG_ERR, "Warning: no UUID: did not processed Sen.se/Zipabox Event: uuid=%s unit: %s, level: %f" % (uuid,unit,level))
56. def eventHandler(subject, content):
57. #print "EventHandler: %s" % (subject)
58. if subject in ("event.security.sensortriggered"):
59. eventHandlerMotion( subject, content)
61. if subject in ("event.environment.temperaturechanged", "event.environment.humiditychanged", "event.environment.brightnesschanged"):
62. eventHandlerMeter( subject, content)
65. # register Event handler
66. client.addEventHandler( eventHandler)

A note regarding my previous post:

Please be aware that what I described does not use my zipabox or the zipato cloud api.

The concerned Zwave devices are directly attached to a Raspberry Pi running agocontrol, which is by the way clearly less easy that attaching devices to the zipabox.

After a few months, by not releasing the feature to send values into http requests (which I consider as a must have and very simple to do with rules) the Zipato team has encouraged me to find an alternate solution ;-)

That's why I consider it relevant to post this story here.

whaou! good job!

Hello,

I first created a doc in French for Open.Sen.se : http://frederic.ravetier-on.fr/2013/10/16/afficher-des-donnees-sous-forme-de-graphiques-avec-open-sen-se-les-objets-connectes/

Coming soon : the script and its documentation

Waiting for this!! Thanks

Finally I decided to publish a project on sourceforge. It includes the code and a wiki page with some explanations. If some other people wants to participate, please ask!

https://sourceforge.net/projects/zipabox-connector

Great!! Thanks a lot (ou plutôt Merci! ;)

Can I use my synology nas as host for these files?

How can I send the informations periodically to sen.se ?

Yes you can.

I did not try but it looks like the cron service is available, have a look to http://lelaboratoiredugeek.com/2013/02/25/tuto-creer-une-tache-planifiee-sur-un-nas-synology/

But I guess if you activated an automatic sleep mode it won't sleep anymore, but it is NAS :), it is its job!

Great Fred! Your the bom!

Yeahhh!!! Thanks Fred!! I'll give it a try right now!!

Keep you updated ;)

Thanks again!

In the config.ini file, is there any space between the = and the sense key?

I have put it like this:sense_key=xxxxxxxxxxxxxxxxxxxx

Concerning the sensors, I put this:

1. [TEMPERATURE]
2. 3ae441f1-bd97-4940-85G7-ecd4e86cj9c4 = 41279
3. Kc843ce5-bd64-4262-9ged-353789209c06 = 41280
4. eP0a3fb7-c66d-428a-8549-fd6Gf8b07889 = 41281
5. [CUMULATIVE_CONSUMPTION]
6. #uuid device or device name = feed id.

Is it the good way to complete the informations?

Which file do I have to execute?

zipabox.sh or zipabox.py?

Thanks in advance

hello

You should ask your question on the page of the project (if it is possible! :D)

I am not using spaces but I think python is removing the spaces...

It looks good.

The shell is nice to run in a crontab but if python is well setup and you set-up executable rights on the file it should work in standalone.

I forgot to say that you need to set executable rights!

I tried but it did not work on my synology ;(

I installed python on it, but I dunno how to check if is well launching the script you created.

I'm launching the script with the task planner of my nas.

When I check the config.ini file, I notice that the password is still there, and no passwordsha1 is created. So I suppose the script was not executed.

I also have modified the zipabox.py file

I have put the files in web directory of my nas, so I have put this:

cd /Volume1/web/sense

python Zipabox.py

I'll post a message on the discussion page of the sourceforge project...

At line 70, replace by:

self.config.write(open(r'config.ini', 'w'))

Then the password will be set in sha1, anyway it is written in the logs so you can copy and paste.

I will publish a new version...

First run it in ssh with the python and you should see logs like:

INFO:Zipabox:Starting Processus...INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): my.zipato.com

Thanks Fred! A few questions.

- I run the script but can't see if it was successfull to sen.se or not.

- What apps are you using in sen.se?

- storing the script on a webserver, how to execute it using a schedule?

- I run the script but can't see if it was successfull to sen.se or not.

There is a log that say: INFO:Zipabox:Data sent to Sen.se

which means it has been sent

- What apps are you using in sen.se?

Mainly MultiViz

- storing the script on a webserver, how to execute it using a schedule?

By using a crontab on linux

Thanks!

- Would it be possible to add current consumption? Since the accumulated i can already follow on dashboard.

- I get an error connecting to sen.se (Attachment)

Hello,

I added the current consumption, just download the last zip and replace the Zipabox.py and have a look to the config example

It was a bug in the script when there is an error during connexion to open.sen.se, it is corrected you will see the error in the logs

Thanks! I have now all 3 values types in the config file. Temp, accum. cons. and current consumption. But still.. For all of them it says...

INFO:Zipabox:NO FEED for Device : xxxxxxx-dc00-418a-ace6-fc5778fb07e8(Refrigerator ) : 0.50

When running the API for the same device uuid and ID it works.

This would be nice...

http://pythonhosted.org/APScheduler/

So i could store the script on a cloud web server and then just let in run.

I found it! There is a bug in the section of the current consumption. Check attachment.

Now it works fine!

You're right! :) bad copye and paste.

For the scheduler, I prefer to have a task running often that a task always running!

Cron is working wonderfully

But you can add a method to do it or a newscript to call this one recurrently !

Any example of a cron python script on server level that can be used?

This code is not using the API, right? Will that cause any performance problems for Zipato?

I do not have any example, I learnt Python when I started to write this script! So my experience is pretty small about python, I am good to Google :)

My code is using the API but I tried to improve it to have few requests to Zipato and when the API will be available in local it will be easy to update, normally it will be only at the config file :).

FYI i set my Mac Mini to a local MAMP running my Php/Python scripts. I then use Lingon X to schedule them. Works like a sharm.

Sen.se is updated with all my meters.

I do have 1 question though, running the python script in dialog it works but upon executing it as a background task it can't find the config.ini file despite its in the same folder.

Did you download the last version of the script?

Because the first one was calling ./config.ini and now directly config.ini. The ./ was not working with a crontab (scheduler)

Yes, I even tried both paths in script wo sucess... :(

Try to display the current path!

And you may need to set the whole path from /

Project Zipabox connector is nice ! I want to install it ! I downloaded file, does it exist some wiki & documentation to install it on the python server which i assume is a server inside my home which is connected to the sen.se ?

Thanks! I set the default folder by using

import os

os chdir('/xxx/xxx'))

Hi Fred,

I'm trying to follow your wiki step by step, a I'm not familiar with linux, pyhton...Where shall I install the files? I did all the steps, but when I type "./zipabox.sh" at the end, I get "-sh: /zipabox.sh : not found".

I've tried with "zipabox.sh", being in the directory where I installed the files ("volume1/homes/admin/Python" on my nas), same result... although I can see the file zipabox.sh making the DIR command in Putty.

Could you help me?

Thanks,

Philippe

Hi

This is not easy to help on this point

It looks like you do not have shell setup!

Try to launch thé command that is inside thé shell

Thanks a lot Fred, I could move a step further.

The command "pyton Zipabox.py" works in my directory "/volume1/homes/admin/Python" : data sent to Sense

I just have to figure out why the command zipabox.sh doesn't work...

Great, It works ! Thanks again Fred, you really did a great job !!!

Fred, Where is located the wiki for your project ? I didn't found link on sourceforge ?

Thanks a lot !

Oups just found it :

http://sourceforge.net/p/zipabox-connector/wiki/Home/

Yep

Thann you

If you sant to improve it, do ont hesitate to stage

Zipato made a good API, also they implemented also graph inside thé dashboard, do ont hesitate to have a look and if necessary to ask them to have a look!

Lol i am using a mobile :)

Do ont= do not

Stage = share

Sant = want

Is anybody using a Synology NAS??

If yes, how do you use this python script on the NAS?

How can I get a log of what is happening when I launch the command?

Hi,

It would be nice to add "humidity" data, and if possible de "luminance". I tried, but I didn't succeed...

I made it modular so it would be easy. I will may have time before thé end of thé month

Fred,

Just updated you project zipabox-connector for the support of up to 3 thermometers, plus the existing meters for power consumption !

this the code :

1. '''
2. Created on 7 oct. 2013
3. @author: Frederic Ravetier
4. Modified by Thierry Bousquet - 16 nov 2013 : add support for 3 thermometers
5. '''
6. import requests
7. import json
8. import hashlib
9. import logging
10. import ConfigParser
11. import sys
12. class Zipabox:
13. """Give methods to access to the data of a Zipabox thru the cloud of Zipato.
14. Provide init, login and logout methods.
15. The configuration file contains the personal informations in order to separate the code from the personnal informations.
16. The password is stored encrypted, so someone can re-use it but can not know the password (that may be used is some others applications...
18. It provides a method to get values from meters depending on feed_id define in the configuration file
19. Basic Usage::
20. >>> zipabox = Zipabox()
21. >>> zipabox.processMetersToSense()
22. """
24. def __init__(self):
25. #### INIT CONFIG ####
26. self.configFile = open(r'config.ini')
27. self.config = ConfigParser.ConfigParser()
28. self.config.readfp(self.configFile)
29. #config.read(".\config.ini")
30. try:
31. loglevel=self.config.get("Development",'LogLevel')
32. except (KeyError, ConfigParser.NoOptionError, ConfigParser.NoSectionError):
33. loglevel="INFO"
34. self.logger = logging.getLogger("Zipabox")
35. if loglevel == "DEBUG":
36. logging.basicConfig(level=logging.DEBUG)
37. else:
38. logging.basicConfig(level=logging.INFO)
39. self.logger.info('Starting Processus...')
41. def initLoginZipato(self):
42. """Initialize the connexion to Zipabox. It initializes the session.
43. :return the data in JSON that contains the nonce. Exists if an error occured.
44. """
45. #### READ CONFIG ABOUT Authentication ####
46. try:
47. self.username = self.config.get("Authentication",'Login')
48. if self.username == '':
49. self.logger.error('Login is missing in the config file')
50. sys.exit(1)
51. try:
52. self.passwordSha1 = self.config.get("Authentication",'PasswordSHA1')
53. except (KeyError, ConfigParser.NoOptionError) as e:
54. self.passwordSha1 = ''
55. #Check if this encrypted password exists. If yes, we use it, otherwise we generate it from the normal password.
56. if self.passwordSha1 == '':
57. password = self.config.get("Authentication",'Password')
58. if password == '':
59. self.logger.error('Password is missing in the config file')
60. sys.exit(1)
61. self.passwordSha1 = hashlib.sha1(password.encode('utf-8')).hexdigest()
62. self.logger.info('You should set the SHA1 password in the config file : '+self.passwordSha1)
63. self.config.set("Authentication",'PasswordSHA1',self.passwordSha1)
64. self.config.set("Authentication",'Password','')
65. self.config.write(open(r'config.ini', 'w'))
66. except (KeyError, ConfigParser.NoOptionError) as e:
67. self.logger.error('An error occured during the initialization of the config File : '+e)
68. #### INIT LOGIN ####
69. try :
70. self.uriZipato = self.config.get("NETWORK",'protocol')+'://'+self.config.get("NETWORK",'ip')+':'+self.config.get("NETWORK",'port')
71. except (KeyError, ConfigParser.NoOptionError) as e:
72. self.uriZipato = 'https://my.zipato.com:443'
73. self.logger.info('Missing NETWORK configuration, cloud configuration by default : '+str(e))
74. path = '/zipato-web/rest/user/init'
75. self.session = requests.Session()
76. self.logger.debug('Init login process to ' + self.uriZipato+path)
77. response = self.session.get(self.uriZipato+path)
78. if (response.status_code != 200) :
79. self.logger.error('Error during init on Zipato : ', response)
80. sys.exit(1)
81. data = response.json()
82. self.logger.debug("response: "+str(response) + ' - ' + str(data))
83. return data
85. def loginZipato(self,data):
86. """Does the login on the Zipabox. The initLogin should be done before
87. :param data : the data in JSON, it should contains the nonce
88. :return 0 if OK; exists otherwise.
89. """
90. #### BUILD TOKEN ####
91. #token = SHA1(nonce + SHA1(password))
92. #sidKey = 'jsessionid'
93. nonceKey = 'nonce'
94. nonceValue = data[nonceKey]
95. #self.logger.debug(sidKey+": "+data[sidKey])
96. self.logger.debug(nonceKey +": "+nonceValue)
97. token = hashlib.sha1((nonceValue + self.passwordSha1).encode('utf-8')).hexdigest()
99. #### LOGIN ####
100. path = '/zipato-web/json/Login'
101. parameters = {'username': self.username, 'password': token, 'method': 'SHA1'}
102. self.logger.debug('Login to ' + self.uriZipato+path + ', params : '+str(parameters))
103. response = self.session.post(self.uriZipato+path,params=parameters)
104. if (response.status_code != 200) :
105. self.logger.error('Error during authentication on Zipato : ', response)
106. sys.exit(1)
107. data = response.json()
108. self.logger.debug("response: "+str(response) + ' - ' + str(data))
109. success = data['success']
110. self.logger.debug('Login Success: '+str(success))
111. if (str(success) != 'True') :
112. self.logger.error('Unable to login : ', data)
113. sys.exit(1)
114. return 0
116. def logoutZipato(self):
117. """Does the logout on the Zipabox.
118. :return 0 if OK; exists otherwise.
119. """
120. path = '/zipato-web/rest/user/logout'
121. response = self.session.get(self.uriZipato+path)
122. if (response.status_code != 200) :
123. self.logger.error('Error during logout on Zipato : ', response)
124. sys.exit(1)
125. data = response.json()
126. self.logger.debug("response: "+str(response) + ' - ' + str(data))
127. success = data['success']
128. self.logger.debug('Logout Success: '+str(success))
129. if (str(success) != 'True') :
130. self.logger.error('Unable to logout : ', data)
131. sys.exit(1)
132. return 0
133. def getFeedId(self,typeDevice,deviceName,deviceId):
134. """Get the feedId from the config file
135. :param deviceName : the device name, not used at this time
136. :param deviceId : the deviceId from the zipabox
137. :return the feed_id matching the device, return an empty string '' if not found
138. """
139. feed_id = ''
140. try :
141. feed_id = self.config.get(typeDevice, deviceName)
142. except (KeyError, ConfigParser.NoOptionError) :
143. try:
144. feed_id = self.config.get(typeDevice, deviceId)
145. except (KeyError, ConfigParser.NoOptionError) :
146. feed_id = ''
147. return feed_id
149. def getTemperature1(self,attributeValue,name, deviceId):
150. #Apply a filter on TEMPERATURE
151. if attributeValue ['definition']['name'] == "TEMPERATURE":
152. value = float(attributeValue['value'])
153. value = "{0:.2f}".format(value)
154. feed_id = self.getFeedId("TEMPERATURE",name,deviceId)
155. if feed_id != '':
156. self.logger.info('feed_id = '+feed_id + ' - Device : ' + deviceId + '(' + name + ') : '+str(value))
157. parametersTmp = {'feed_id': str(feed_id), 'value': str(value)}
158. return parametersTmp
159. else:
160. self.logger.info('NO FEED for Device : ' + deviceId + '(' + name + ') : '+str(value))
161. def getTemperature2(self,attributeValue,name, deviceId):
162. #Apply a filter on TEMPERATURE2
163. if attributeValue ['definition']['name'] == "TEMPERATURE":
164. value = float(attributeValue['value'])
165. value = "{0:.2f}".format(value)
166. feed_id = self.getFeedId("TEMPERATURE",name,deviceId)
167. if feed_id != '':
168. self.logger.info('feed_id = '+feed_id + ' - Device : ' + deviceId + '(' + name + ') : '+str(value))
169. parametersTmp = {'feed_id': str(feed_id), 'value': str(value)}
170. return parametersTmp
171. else:
172. self.logger.info('NO FEED for Device : ' + deviceId + '(' + name + ') : '+str(value))
173. def getTemperature3(self,attributeValue,name, deviceId):
174. #Apply a filter on TEMPERATURE3
175. if attributeValue ['definition']['name'] == "TEMPERATURE":
176. value = float(attributeValue['value'])
177. value = "{0:.2f}".format(value)
178. feed_id = self.getFeedId("TEMPERATURE",name,deviceId)
179. if feed_id != '':
180. self.logger.info('feed_id = '+feed_id + ' - Device : ' + deviceId + '(' + name + ') : '+str(value))
181. parametersTmp = {'feed_id': str(feed_id), 'value': str(value)}
182. return parametersTmp
183. else:
184. self.logger.info('NO FEED for Device : ' + deviceId + '(' + name + ') : '+str(value))
185. def getCumulativeConsumption(self,attributeValue,name, deviceId):
186. #Apply a filter on CUMULATIVE_CONSUMPTION
187. if attributeValue ['definition']['name'] == "CUMULATIVE_CONSUMPTION":
188. value = float(attributeValue['value'])
189. value = "{0:.2f}".format(value)
190. feed_id = self.getFeedId("CUMULATIVE_CONSUMPTION",name,deviceId)
191. if feed_id != '':
192. self.logger.info('feed_id = '+feed_id + ' - Device : ' + deviceId + '(' + name + ') : '+str(value))
193. parametersTmp = {'feed_id': str(feed_id), 'value': str(value)}
194. return parametersTmp
195. else:
196. self.logger.info('NO FEED for Device : ' + deviceId + '(' + name + ') : '+str(value))
198. def getCurrentConsumption(self,attributeValue,name, deviceId):
199. #Apply a filter on CUMULATIVE_CONSUMPTION
200. if attributeValue ['definition']['name'] == "CURRENT_CONSUMPTION":
201. value = float(attributeValue['value'])
202. value = "{0:.2f}".format(value)
203. feed_id = self.getFeedId("CURRENT_CONSUMPTION",name,deviceId)
204. if feed_id != '':
205. self.logger.info('feed_id = '+feed_id + ' - Device : ' + deviceId + '(' + name + ') : '+str(value))
206. parametersTmp = {'feed_id': str(feed_id), 'value': str(value)}
207. return parametersTmp
208. else:
209. self.logger.info('NO FEED for Device : ' + deviceId + '(' + name + ') : '+str(value))
210. def getMeters(self):
211. """Get the Meters from the Zipabox. At this time it gets only the temperature of the devices in the config file.
212. :return the data in JSON with feed_id and values, return [] if nothing
213. """
214. #Browse the meters
215. #/zipato-web/rest/meters/
216. path = '/zipato-web/rest/meters/'
217. self.logger.debug('Get meters: ' + self.uriZipato+path)
218. response = self.session.get(self.uriZipato+path)
219. if (response.status_code != 200) :
220. self.logger.error('Unable to get meters from zipato : ', path, response)
221. sys.exit(1)
222. data = response.json()
223. self.logger.debug("response: "+str(response) + ' - ' + str(data))
224. parameters = []
225. for deviceId, deviceValue in data.items():
226. name=deviceValue['name']
227. try :
228. offline=deviceValue['offline']
229. except (KeyError) :
230. offline = False
231. attributes=deviceValue['attributes']
232. for attributeKey, attributeValue in attributes.items():
233. self.logger.debug('name = ' +attributeKey + ':' + str(attributeValue))
234. parametersTmp = self.getTemperature1(attributeValue,name,deviceId)
235. if parametersTmp:
236. parameters.append(parametersTmp)
237. parametersTmp = self.getTemperature2(attributeValue,name,deviceId)
238. if parametersTmp:
239. parameters.append(parametersTmp)
240. parametersTmp = self.getTemperature3(attributeValue,name,deviceId)
241. if parametersTmp:
242. parameters.append(parametersTmp)
243. parametersTmp = self.getCumulativeConsumption(attributeValue,name,deviceId)
244. if parametersTmp:
245. parameters.append(parametersTmp)
246. parametersTmp = self.getCurrentConsumption(attributeValue,name,deviceId)
247. if parametersTmp:
248. parameters.append(parametersTmp)
249. feed_id = self.getFeedId("Sen.se",'offline_feed',deviceId)
250. if offline and feed_id != '':
251. self.logger.info('feed_id = '+feed_id + ' - Device : ' + deviceId + '(' + name + ') : Offline : '+str(offline))
252. try:
253. parametersTmp = {'feed_id': str(feed_id), 'value': name.encode('utf-8')}
254. #Do not insert doubles, some devices are able to manage more than one 'device'
255. if parameters.count(parametersTmp) == 0 :
256. parameters.append(parametersTmp)
257. except (UnicodeEncodeError) as e:
258. self.logger.error(e)
260. #parameters = {'feed_id': str(feed_id), 'value': str(value)}
261. return parameters
263. def sendToSense(self,data):
264. """Send the data to Sen.se. The sens.se key is get from the config file
265. :param data : the data to send. It should be a list of dict
266. :return 0 if success, 1 otherwise. return 0 if there is no data to send
267. """
268. self.logger.debug("data : "+str(data))
269. if data:
270. senseKey = ''
271. uriSense = "http://api.sen.se";
272. if senseKey == '':
273. senseKey = self.config.get("Sen.se",'sense_key')
274. path = '/events/?sense_key='+senseKey
275. headersSense = {'Content-type': 'application/json'}
276. self.logger.debug('Send data ' + uriSense+path + ', params : '+json.dumps(data))
277. response = requests.post(uriSense+path,json.dumps(data),headers=headersSense)
278. if (response.status_code != 200) :
279. self.logger.error('Error sending data to sen.se : '+ str(response))
280. self.logger.debug("response: "+str(response))
281. return 1
282. else:
283. self.logger.info('Data sent to Sen.se')
284. return 0
285. return 0
287. def processMetersToSense(self):
288. """It does the whole process: init, login, getTemperatures, sendToSense.
289. """
290. data = self.initLoginZipato()
291. result = self.loginZipato(data)
292. if result == 0:
293. try:
294. parameters = self.getMeters()
295. #self.logger.info(str(parameters))
296. self.sendToSense(parameters)
297. finally:
298. self.logoutZipato()
301. if __name__ == '__main__':
302. zipabox = Zipabox()
303. zipabox.processMetersToSense()

and ini file :

Could you please send me the code as an attachment, I will do a diff and see if there is a better solution to implement that. On this thread or by the http://sourceforge.net/users/fravetier

I added humidity and luminance in my code, but I will publish everything with the temperature

many thanks and great work!

if like me you do not have a Sinology NAS, a Raspberry or another linux stuff to put and launch the zipabox conector script (i know that i am not the only one)

I have a solution !

you can use an internet service provider like alwaysdata.com (but you can find many others like this one)

It is free and provide an ftp to drop the script and a ssh to configure the crontab job !

you can now pool your sensor in 24/24 :-)

@Olfo Great tip!

Can it also work with google drive/scripts?

Do sen.se only store a max number of data or why can't i analyse more then 1 day?

It is strange because on the graph where I use 3 datas I have 2 days back displayed, on where I use only 1 data I have 4 days back on the graph.

When I look the events of a device and I ask for more, I can go back many days ago.

From my experience it seems that sen.se keeps all values (I've got around 6 months of data still available exploring the feeds) BUT the graphs also seem to be limited (I would say according to the last xxxx values, which explains it's does not represent always the same duration).

@HeroS i'm sorry i don't know how google drive/script look's like but to do that you need to write another script in php, python or something's like this to remotely launch in google drive your script (it's going to be difficult)

i'm not sure but i think google drive dis delivering a schedulling box?

Hi Olfo,

Noproblem. I looked at Google but is to complex for me. I got an account on alwaysdata.com.

Can u tell me what i need to change to get it working and how to make the cron job? Already tried something and got the script running trough the command line.

But it starts with no such file or directory (i put the files in www) and i don't see the connection to sense.

Thanks!

I updated the module with HUMIDITY and LUMINANCE

I added a new module to have a virtual sensor for DAY/NIGHT depending on SUN without any external website ;).

I am still waiting for Thierry Bousquet to get the file by mail otherwise it is not easy to compare!

regarding alwaysdata

Here the translation of my french topic :

http://www.touteladomotique.com/forum/viewtopic.php?f=122&t=12522

1/ create a free account

2/ activate SSH on control panel

3/ upload all the files (after update of config.ini) in the HOME directory using FTP (or using web FTP app : https://net2ftp.alwaysdata.com/ )

You can also upload directly .zip file. It will be unzip automaticly

4/ using SSH (putty or web : https://ssh.alwaysdata.com/anyterm.html)

- launch script using command "python ./Zipabox.py"

- you will see all the zipabox sensor ID

- download config.ini using FTP (Zipabox password is now crypted). Update zipa id = sen.se feed Id & reupload

- launch "python ./zipabox.py". Normaly you should see datas in Open.sen.se

5/ Setup Cron using SSH

- command : "crontab -e"

- command "*/15 * * * * python ./Zipabox.py" to launch script every 15 minutes. You can change it please find exemples using this link http://wikipedia.org/wiki/Crontab

- CMD + X to exit and save

Enjoy !

Je teste ça ce soir!!! Merci Tony et Fred ;)

Du coup pas besoin de serveur pour héberger le script?? Il est lancé directement sur le site de alwaysdata?

Alwaysdata est l'hebergeur du serveur justement

Oui je viens de voir ça, je suis en train de lire le tout, mais je bute sur un message d'erreur:

1. Traceback (most recent call last): File "./Zipabox.py", line 7, in <module> import requestsImportError: No module named requests

Hi Tony thanks!

But i'm not seeing any data come in at sense but the script is running.

When i run the script is says:

INFO:Zipabox:NO FEED for Device :

Do i need to do anything else?

J'ai tout repris...c'est bon ça fonctionne! ;)

J'ai bien mes données sur Sen.seMaintenant je vais tenter de voir voir comment lancer le script directement de mon nas...

Fred created a tuto for open.sen.se

But it's in french

http://frederic.ravetier-on.fr/2013/10/16/afficher-des-donnees-sous-forme-de-graphiques-avec-open-sen-se-les-objets-connectes/#axzz2lsRInVfE

You need to create device and feeds in open.sen.se

When you run the python script it will show you all the zipabox feed avalaible

edit again confi.ini after first load to put

id.feeed.zipabo = id feed open.sen

Ahh, got it now! Super Super thanks!!

Thank you, I will see what I can do

Thierry, are you sure about your code? For me you get three times the same value!.

Can you contact me on touteladomotique (fredblabla)?

No it works fine, you should have inside ini file 3 sections [Temperature], the difference between the temperature device is made by their ID. If it is your question ?

I tried using sections [TEMPERATURE1], [TEMPERATURE2], ... but it was NOK.

Ok so your code is not working but you are close :)

No it works using 3 sections [TEMPERATURE] inside ini file.

It works for me too using alwaysdata solution...

I'm ok with it...I'll see later what I can do with my NAS.Thanks again!

Great, i will try to optimize using only one section temperature with 3 devices declared inside as Fred suggested.

Fred, ok using 1 section like this it works fine !

[TEMPERATURE]

35919673-c20e-4299-a862-220d841b1b96 = num_feed_id1

5b58daf26-b0f8-4e2d-bc2-52 5222b4c32 = num_feed_id2

11c222-a4af-405a-9d36-20bfd045555b55 = num_feed_id3

I added a new functionnality:

The request to the sensor is sent only if the sensor has to be modified.

You need to update Zipabox.py and Sunsensor.py and to add something in the config.ini. If the sensor uuid is not defined, the script will list the sensors, so it is easy to add the uuid ;)

"Le graphique de Multiviz ne montre que les dernières 1000 données dans tous

les feeds combinés parce qu'afficher des données sur des périodes plus longues

reviendrait à aller chercher des dizaines voire des centaines de milliers de

valeurs à chaque affichage de la page.

Deux façons d'afficher des données sur des périodes plus longues:

- Envoyer des données moins souvent: (quand vous voulez afficher des

graphiques sur 1 an, des fluctuations sur 5 minutes seront invisibles.

- Utiliser l'application Count Sum & Average qui va créer des

agrégations de vos donnés, par exemple en calculant une moyenne horaire,

hebdomadaire, mensuelle etc. Ensuite vous pourrez faire le graphique des

résultats de Count Sum & Average. Par exemple si vous faites une moyenne

quotidienne de vos données, vous pourrez grapher les 1000 derniers jours. "

I stopped using Sense.se, I 'm currently trying another box that directly gives me charts...

I corrected a bug (new release) on my last improvements

@Phillip thank you for the info. What box are you using?

Do you guys think this is enough graphs for 3 temp. meters :)

Nop but it is free

Just installed zipabox connector it works fine i've now luminance and humidity tracked, for the Solar features what's for Fred ?

You mean Sun feature. The goal is to have a virtual sensor that act as a SUN sensor. It is activate when the appears (in the morning for the sunrise) and inactivate at the sunset for the night.

It is cool to open or close store for examples. It does not use any external website. It is processing with an algorithm and it depends on your location, that's it.

@Fred: for the sun feature, I am using ifttt and google drive, but your script seems great too.

Is it necessary to launch it periodically with cron?

Because I get a notification when sun goes up/down, and I suppose I will get a notification every time the script is launched...

@thomasfr : so you need internet to do it ;)

yes, I do it with a cron, each time you run the application, it checks the sun and update your virtual device only if necessary. I do it every 5 minutes at home and I set-up a cron only at the hours where it is possible. I wrote a tip on the sourforge for setting the cron.

When I release for the firt time this update, the notification was at every call but then I did an update to set the notification only if necessary, the app get the state of the device and toggle if necessary.

I will try the new version

Merci pour ton aide! ;)

Bonjour,

j'ai mis en place la relève de mes consommations. Avant de m'attaquer aux graphes, j'aimerais maintenant y adjoindre les relevés de température.

Mais je me heurte à la procédure de connexion à la zipabox à laquelle je ne comprends rien: construction du token notamment.

Quelqu'un aurait-il des exemples de scripts et programmes en python?

Merci par avance.

============================================

Hello,

I've put over my consumption. Before I address the graphs, I would now add there the temperature readings.

But I hit the login procedure to zipabox which I do not understand: construction of particular token.

Does anyone have examples of python scripts and programs?

Thank you.

Hello,

If you look at this post I did a program with python to get the temperatures and send data to sense.se.

So you can use it or download the code and see how the token is working :)

thank you,

I had already seen, but it seemed too complicated to me because I just wanted to initialize my connection to the Zipabox. I thought I would come back later.

I will look in more detail.

Which version of Python did you use?

If I have questions, I will use sourceforge.

Hi Fred,

I just thought it would be nice to display a histogramm chart that displays per day the CumulativeConsumption.

For exemple : Day 1=5Kw/H, Day 2=2KwH, ...

Theses infos are extracted from daily CumulativeConsumption.

Calculation is done by difference between CumulativeConsumption for :

day N+1 - day N at 0:00 each day.

I can help to do that, can you tell me how to do that ?

What could be the principle of the modification in your code ?

For sure we need to store in a variable or file, the N-1 CumulativeConsumption then once a day do the calculation ?

Any ideas are welcome :-)

Hello Thierry,

It is funny because these last days I was trying to find a solution to do that.

I wanted to get some graphs like the one I got from the energy provider, that are nice. But at this time we got some useless graphs, as the one from the zipabox about consumption (not the one about temperature).

I had a look to the events but I do not understand how it works!.

I posted a message on our french forum :

http://www.touteladomotique.com/forum/viewtopic.php?f=115&t=12783

At this time my progam is working to get data every xx minutes. But I do not think that this strategy is good for doing that. But I do not know which one is good. May be to retrieve events from the box...

If you have an idea to get the real datas.

Do you think that every event is a cumulative consumption and that after each events it start to 0 ?!

Then we will find an algorythm...

This is a guide on eedomus for what you are looking to do with zipabox, maybe it can help:

http://forum.eedomus.com/viewtopic.php?f=12&t=1340

Your link is interesting but it does not explain how to get the values about consumption correctly

I need to understand how it works

Thierry's hint:

(cumulate consumption dayN-1) - (cumulate consumption dayN) = consumption of the day

could do the trick, doesn't it?

But how do you get these values from the zipabox?

Have a look to the screenshot I've done. The value are not per day. The device don't know the time.

I haven't took enough time on the Api and the events.

I really cannot help you on that one ;(

Definitely :

cumulate consumption dayN-1) - (cumulate consumption dayN) = consumption of the day

could do the trick, what we can do it's the following :

1. Duplicate instance of my zipabox let's call it Zipabox_Day

This instance has only one feed_id which takes Cumulative_Consumption.

It stores value in one file let's say power_cons_day.txt

2. Add a new cron that launch this new instance each day at 0:00

3. We can have now a bar graph in sense that display evolution of cumulative power consumption per day

If we modify instance : Zipabox_Day to do that : at start_up load value of last cumulative_power_consumption and store in a variable, then get new value, then calculate difference, then send this value we could have the power consumption per day :-)

I already tested to duplicate instance and it works, now i think we have to manage the generation of the file that countains the power consumption and so on ?

Fred : what do you think about it ?

Answer to Fred for :

"But how do you get these values from the zipabox?

Have a look to the screenshot I've done. The value are not per day. The device don't know the time."

=> we need to Get one time per day the cumulative consumption (see my previous post)

=> this is not Zipato that generates the cumulation this is the meter itself and we can get it with your nice software !

also you sayed :

"Do you think that every event is a cumulative consumption and that after each events it start to 0 ?!"

=> every event is a cumulative consumption not starting 0 and in between each event meter sum the real time value : it's a counter of energy like your EDF counter. That's why doing the difference between cumulative power consumption between N Times gives the total consumption for N times.

I think you are missing some data

Have a look to the events on my screenshot

http://imageshack.com/a/img69/8277/y540.jpg

It starts back to zero! At a random time or when it is turn off. Or I missed something!

I think that's depend on your meter, my meter : Aeon Home Energy Meter DSB09104 is always increasing like my EDF counter ! Even if i change battery or if i power off it restart from last previous value then count count ... What is your meter ?

Most of my radiators are using "Aeon micro SES G2". and I have one G1.

And I do not understand how it works.

Actually a meter like your would be easier to follow and you're idea is good. A database would be better. But did you try with sens.se because they give you some applications to combine and do some process on data.

Yes i searched but didn't found it, may be I can ask their support because now I have only 1 value per day of cumulative consumption becausse of the 2 instance Zipabox the only things which is missing is to do the difference.

So I had a look to the events (see my screenshot on the french forum).

The events occurs every 12 minutes (720 seconds) and it starts from ON.

I wrote to Aeon to have more informations.

Have a look to the app on Sen.se called "Count, Sum, Average" and try "the variation from the previous value".

It can't work for me because I would need to get the value exactly when the device send it.

I post a message on this forum to get the events from the API:

http://community.zipato.com/responses/how-to-get-the-events-from-the-api

>Have a look to the app on Sen.se called "Count, Sum, Average" and try "the > >variation from the previous value".

but how can you build a histogram graph as requested ?

I know, I tried since my device are giving a real cumulative consumption, but it does not work!

The only solution is to have something good on zipabox or to have a database as suggested before. Which means having time to make it, which is not my case for now.

I've got it !

I duplicated instance of my Zipabox.

Now a new instance of my zipabox publish cumulative consumption one time per day at 0:00.

Then i'm able (as you suggested) to get by the variation information requested using appi sum, average,variation. Then i'm able to build a graph that show difference of power consumption per day. Also i'm able to calculate in euro how much i consume per day using expressive signs. I will publish later some capture screen once i will have more datas retrieved and that show on screen theses features.

Great & welld one thierry!

Why the behavior is different when duplicating the instance?

How have you done that?

And in a general matter, what's the goal of have a second instance?

Regards

Fred

With the first instance of Zipabox, i send datas every 15 minutes because i send also Temp, humidity, etc... Then if i use application sum, average, variation to calculate variation of cumulative consumption i will have variation of cumulative consumption versus previous value 15 minutes ago. This is not what i want ! I want variation for day. That's why i duplicated Zipabox to create an instance that publish every 24 hour only cumulative consumption. Then i can calculate variation for 24 hours datas. I hope i'm clear.

To duplicate the instance, i copied in an other directory the original Zipabox file directory, then modified config.ini and let only cumulative consumption to be sent.I also created a new arduino object in open.se. Then i created a cron that run every 24 hour at @0:00. And it's done.

ok :)

I though it was on the zipabox.

You created an new instance of the python script :). That was a good idea.

Also the script can bu updated to take some parameters to choose what to send. So then it is just the right command to call in the crontab!

>Also the script can bu updated to take some parameters to choose what to send.

Please explain what do you mean ?

I mean that it requires some lines of codes but it would be nice to call the script like :

./zipabox.sh CUMULATIVE_CONSUMPTION and then it process only the cumulative consumption...

I will try to find some time to do it!

It is done, Have a look to the doc : https://sourceforge.net/p/zipabox-connector/wiki/Home/

Download the zip, extract and replace the zipabox.py.

Then you can have a look to the shell zipaboxCumulative...sh it gives you an example. You can add one line in your crontab to run the script every day on cumulative consumption.

I'll try this new update!! Great ;)

I suppose we have to add a new sensor in sen.se too?

What is the command in crontab?

this one does not work:

0 0 * * * python ./ZipaboxCumulativeConsumption.sh

So it worked? I had to remove on sense the first value sent because it was to big compare to the diff every day and the graph had a first data big and then it looked like the next values were null (0).

Do

0 0 * * * /home/..../ZipaboxCumulativeConsumption.sh

Replace the .... by the path

I just realized I was using python command but it is not a python script...am I right?

That's a stupid error from me ;)

I'm really new with that kind of stuff.

So I have to create one sense graph for cumulativeconsumption of one of my plugs (and ad it's uuid in config.ini), then create another sense graph for the ZipaboxCumulativeConsumption.sh script, that's it?

I'll give it a try this we, thanks

Hello, this is a capture screen of cumulative consumption generated with 2 instances of Zipabox. That's great you modified to have it running in one instance, i will try to use it !

ps : that's a pity i do not receive any notifications for new post message in this feed even if i subscribed so i didn't read you since 1 week ...

and impossible to publish a capture screen even if it is a jpeg < 2MO ... :-(

Hi Fred,

It is really an amazing work. I am currently converting your python code to Google Script (I don't have a NAS...) and it is working pretty well :)

I hope I will be able to show the sources once it will be completed if someone is interested.

Yeah GREAT ...me want me want :)

Thank you. If you want to publish your code on this sourceforge, you are welcome, I can give you an access to write.

Google Script, I am interested.

I didn't know so many people will be interested :)

The code is not complete yet and I only have a temperature meter so I am only be able to test this feature...

I'm doing it during my free time so I will try to release something as soon as possible.

As promised, I released a google script version of the connector.

I hope you will be able to use it as it was made first for my own needs. It is also my first time using google script :)

https://docs.google.com/spreadsheets/d/1iaMnRGPozVS3QaXkUDiwacR-g_7CS4xLsEsy-ew65aA/edit?usp=sharing

1. Make a copy of the file on your google drive,
2. Put your data in the params sheet
3. Go to the script editor and execute the "main" function

You can also use the "Zipabox menu" on the spreadsheet. First launch "Prepare all data for sense" and then launch "Send feeds to sense".

Please don't hesitate to modify the code and share it with other zipabox users !

Great Great Great! Going to give it a spin asp.

You will see, values are also stored in the spreadsheet (a sheet is automatically created for each device before sending feeds to sense) so you can use them to make your own charts :)

Ok great! done some testing...

But not working :( Getting this error:

TypeError: Cannot call method "getLastRow" of null.

Line: 67

Oh I see, you have to add a value in project properties (see screenshot) :

File -> Project properties -> Project properties

name : "paramSheet"

value : "Paramètres" => it is the tab name of params data

I just added it in the spreadsheet so you can copy it again if you need to.

Your the best!

Values are coming in. I have put a trigger on it for more testing.

Thanks again.

I will add a few improvements very soon (like a RSS flow or other things I don't know yet...)

I was celebrating a bit to fast. It only runs sometimes and not all UUID end points work. I see the UUID in the log.

I'm getting this error:

Trop d'appels effectués en peu de temps pour ce service : properties. Essayez Utilities.sleep(1000) entre les

And this one:

Your input contains more than the maximum of 50000 characters in a single cell.

Can not get my finger on it what the problem is. Going to do more testing and get back to you.

The log file should be too big to fit in a single cell of the spreadsheet (you should have a lot of devices!), I will disable logging in the spreadsheet for now (it will still be available in the script editor).

About the properties error I didn't know calls to to PropertiesServices were limited (it seems like your cron is executing the script very often!). I will try to correct this point tonight.

Oke great.

I have 82 end points. But it is strange that is runs sometimes and when i count the char. It is "only" 43500.

I put the trigger to run every 5 min.

I updated the spreadsheet so everything should be OK now.

You can copy it again to test.

Maybe you could raise issues in my new github page so it will be much easier to follow:

https://github.com/ntrinh/zipabox-gs-connector

Sorry for my low level in coding, but could you add a step by step tutorial on your github please? ;)

I did a copy of your zipabox gs connector, but I get a spreadsheet (like excel).

I modified parameters.

Add "paramSheet" and "Paramètres" (in script editor)

The script is executed when preparing all datas, but then nothing is received in sense when I click send datas to sense

Also there is no data in Door/Sensor logs

Which script do I need to schedule?

Thanks for your hard work (Merci!)

Did you rename the tabs of the spreadsheet? (make sure the names match)

And also check the log (see script editor) and try running the "main" from the script editor (not from the spreadsheet).

HeroS is right, the best way now is to go in the script editor and to run the "main" function. I only have a door sensor and a temperature meter so I cannot test every devices. That is why I appreciate your help for testing :)

Don't forget that the parameters "paramSheet" has to be put in the "project properties" tab and not in the "user properties tab".

HeroS is right, the best way now is to go in the script editor and to run the "main" function. I only have a door sensor and a temperature meter so I cannot test every devices. That is why I appreciate your help for testing :)

Don't forget that the parameters "paramSheet" has to be put in the "project properties" tab and not in the "user properties tab".

Ok I get the values, but not the good ones ;)

For example, with my aeon multisensor, I get a MOTION/NO MOTION data, not temperature

For my temp/humidity sensor, I get the humidity number

I checked the uuids and they seem good (the same used with my config with alwaysdata)