Steven M Castano Posted November 28, 2016 Author Posted November 28, 2016 Steven: as far as this step goes: can you expound on how to do this using GitHub, HeidiSQL, MariaDB and Python? I don't understand what you mean here. thanks again, Steven. With HeidieSQL you would connect to your database, so on your local host, and you can just create a blank database using the gui. Name it anything you'd like and use the name and the username and password you used in HeidiSQL to connect in the iphonelocation.ini file as your database setting. Your hostname would be either "localhost" or "127.0.0.1" then and the standard port of 3306 should be fine.
someguy Posted November 28, 2016 Posted November 28, 2016 Okay, I have a database created in HeidiSQL. What next?
memphis2k Posted November 29, 2016 Posted November 29, 2016 Steven, A while back in the thread, you talked about the script you use on your ASUS router. I have hacked together a script in PHP that will "ping" devices and displays a result if the device is alive. Whats your thoughts on something like this as opposed to the ARP script / Disconnect one you have. Battery life etc. I do have an ASUS AC1900 RT-AC68U Router configured as an Access-Point only. I run SOPHOS firewall in a VM in a Server20212 R2. I do have three RasPi's too, 1-3. I have PHP and Python running on the Server too. I'm willing to mod the Wifi router too... Knowing what I use, just looking for the best approach on monitoring my devices when they are home and changing the values for this script. Thanks for your .02 ~Aaron
Steven M Castano Posted December 12, 2016 Author Posted December 12, 2016 Okay, I have a database created in HeidiSQL. What next? Capture.JPG Sorry, I vanished there for a bit... I've had a few crazy weeks. Now that you've created the database, you just put the database name and the mysql login and password in the iphonelocation.ini file and the script will take care of the rest of creating the tables and stuff you need. Then you can fill in your iCloud account instructions and keep following the instructions on how to list devices and select the one you want to use for GPS location.
Steven M Castano Posted December 12, 2016 Author Posted December 12, 2016 Steven, A while back in the thread, you talked about the script you use on your ASUS router. I have hacked together a script in PHP that will "ping" devices and displays a result if the device is alive. Whats your thoughts on something like this as opposed to the ARP script / Disconnect one you have. Battery life etc. I do have an ASUS AC1900 RT-AC68U Router configured as an Access-Point only. I run SOPHOS firewall in a VM in a Server20212 R2. I do have three RasPi's too, 1-3. I have PHP and Python running on the Server too. I'm willing to mod the Wifi router too... Knowing what I use, just looking for the best approach on monitoring my devices when they are home and changing the values for this script. Thanks for your .02 ~Aaron I tried doing this for a while myself and found there's 2 major problems: 1) The ping response is just as flaky as the device staying on wifi. It won't respond to pings a lot of the times even when it's there... so you're still stuck with the same problem with the ARP method in that you have to account for temporary disconnects 2) Since a ping is an active part of the TCP/IP stack, the phone itself has to create the ping response and send data back. So you're actually asking the phone to do work. Work = battery = does dead faster. The ARP method is totally passive and really doesn't depend on the iOS device at all, so you still have problem number 1, but you don't even negatively effect battery life. I would still point you in the direction of a passive monitor. If you've got any way of writing a script on the firewall or mod'ing the router so you can run a bash script, then those are probably your best bets. You just have to make sure you're monitoring things from a location that ALL wifi traffic has to pass through. If you only have the one ASUS router as your only WiFi access point, that might be the perfect place to do it. Run OpenWRT on it and I can clean up and post the bash script I use to look at the ARP table.
memphis2k Posted December 12, 2016 Posted December 12, 2016 I tried doing this for a while myself and found there's 2 major problems: 1) The ping response is just as flaky as the device staying on wifi. It won't respond to pings a lot of the times even when it's there... so you're still stuck with the same problem with the ARP method in that you have to account for temporary disconnects 2) Since a ping is an active part of the TCP/IP stack, the phone itself has to create the ping response and send data back. So you're actually asking the phone to do work. Work = battery = does dead faster. The ARP method is totally passive and really doesn't depend on the iOS device at all, so you still have problem number 1, but you don't even negatively effect battery life. I would still point you in the direction of a passive monitor. If you've got any way of writing a script on the firewall or mod'ing the router so you can run a bash script, then those are probably your best bets. You just have to make sure you're monitoring things from a location that ALL wifi traffic has to pass through. If you only have the one ASUS router as your only WiFi access point, that might be the perfect place to do it. Run OpenWRT on it and I can clean up and post the bash script I use to look at the ARP table. Steven & ALL others wondering on the Proximity script for there routers. This script can be modified to work on DD-WRT and OpenWRT and ASUS Merlin firmware. Please change the below to fit your configuration. Here is the script that I am running on my ASUS RT-AC68U. I am running the latest version of the Merlin firmware designed for the router. See below. I spoke with the creator of the Merlin script and he gave me some advice on the script. Use "arpout=$(wl -i eth2 assoclist)" INSTREAD of the "arp -i br0". At least for the Merilin firmware as it found the devices even when they would fall asleep. I have further tweaking ahead since I'm moving away from MobiLinc for my locations, but this morning provided to be a success on my way to work. #!/bin/sh ###################################################################################################################### # Proximity detection # # # # A script designed to run on a router running DD-WRT to detect certain devices connected to the router. # # It runs at startup and runs continually, checking for a specific list of devices (phones/laptop, etc) # # that connect wirelessly to the router. Once a device is connected, the status is updated on the ISY # # with either a 0=disconnected, or a 1=connected. State variables are used on the ISY so that a change # # in status (0/1) can trigger a programe on the ISY. # # # # The searching frequency can be adjusted to be slower/faster depending on your requirements. Searching too fast # # could burden your router. Too slow might not update the status as necessary for the application. # ###################################################################################################################### # Make changes below # This variable turns on or off notifications to the syslog for debugging. # Set 'notifications' to '1' to enable logging, set it to '0' to turn them off. notifications=1 # Logging is enable by default. Change this to turn it off. # MAC address of each device to watch. Don't leave blank. All CAPS on MAC Letters if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Setting MACs"; fi #Debug output to the syslog. macdevice1="****" #Aarons iPhone macdevice2="****" #Rachels iPhone macdevice3="**:**:**:**" #Aarons iPad if [ $notifications -eq 1 ]; then #Debug output to the syslog. # logger -t PX_DETECT "Set MACs:" #Debug output to the syslog. logger -t PX_DETECT "macdevice1=" $macdevice1 #Debug output to the syslog. logger -t PX_DETECT "macdevice2=" $macdevice2 #Debug output to the syslog. logger -t PX_DETECT "macdevice3=" $macdevice3 #Debug output to the syslog. fi # if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Setting ISY username, password and local IP"; fi #Debug output to the syslog. # ISY username, password, and IP Address username="admin" password="****" IPAddr="192.168.1.***:***" if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Set ISY username, password and local IP"; fi #Debug output to the syslog. # State variable numbers on the ISY corresponding to each MAC above # isyvartot is the cummulative number of devices (of the 4 above) that are # currently connected. isyvar1=12 isyvar2=19 #CHANGE TO 5 ONCE THE OTHER SCRIPT SUPPORTS MULTIPLE USERS isyvar3=43 isyvartot=22 # Occupied and unoccupied delay in seconds to check status # Adjust for shorter/longer wait times. For instance, when one device is already # connected, you might want to check less frequently. This could also delay the # notification of a disconnect. delay_occupied=4 delay_unoccupied=2 # initial testing loop count - uncomment the counter near the bottom of the script for testing only. limit=5 # if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Testing loop limit: " $limit; fi #Debug output to the syslog. ############################################### # do not change below here ############################################### # DO NOT CHANGE THE LINE BELOW. THE "XXXX" IS WHAT *SHOULD& BE THERE!!! # This creates a variable use to store the output from a single arp command # so it can be tested multiple times for the presence of any of your devices. # It is used to lower the number of times the arp command is run to a single # time per test cycle. It also store the output in RAM as opposed to using # additional resources to write it out to disk/nvram. arpout=XXXX sleep #initialize internal variables if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Setting connection status to initial state of -1"; fi #Debug output to the syslog. # status of each MAC. 0=disconnected. 1=connected. -1 initially forces isy update first loop macconnected1=-1 macconnected2=-1 macconnected3=-1 connected=-1 # total number of currently conencted devices. currentconnected=0 # initializing the counter at 1 counter=1 # initializing the interval counter to be used to output less frequently # to the syslog service during the loop. intervalcounter=1 if [ $notifications -eq 1 ]; then logger -t PX_DETECT "All variables and controls are set, beginning script logic/testing loop"; fi #Debug output to the syslog. # Initial testing loop. Will run continually after testing is complete while [ $counter -lt $limit ]; do #if the interval counter is 1 then make a log entry. Uncomment to show Interval loop count status # if [ $notifications -eq 1 ]; then # if [ $intervalcounter -eq 1 ]; then # logger -t PX_DETECT "Test loop " $counter " of " $limit "is running." # fi # fi # Reset current status. Two variables are used for each device. The past known status and the current # status. Only a change is reported to the ISY. Otherwise, it would constantly be updating the ISY with # the current status creating unnecessary traffic for both the router and the ISY maccurrent1=0; maccurrent2=0; maccurrent3=0; # This section compares the list of MACs you entered above against the output of a # single arp command listing the MACs associated with the wired bridge of your # router. Interface "br0" # This command runs arp once and saves it in the 'arpout' variable in memory - Old command is (arp -i br0) arpout=$(wl -i eth2 assoclist) # The following runs 'grep' against the output form 'arp' in order to see if your # first entered MAC address is present on the network. maccurrent1=$(echo $arpout | grep -c $macdevice1) if [ $maccurrent1 -gt 0 ]; then maccurrent1=1 fi # The following section checks the second MAC identical to method discussed in the # section directly above. To check additional MACs this section can be copied and # the numbers following 'maccurrent' and 'macdevice' should in incremented. maccurrent2=$(echo $arpout | grep -c $macdevice2) if [ $maccurrent2 -gt 0 ]; then maccurrent2=1 fi # The following section checks the third MAC identical to method discussed in the # section directly above. To check additional MACs this section can be copied and # the numbers following 'maccurrent' and 'macdevice' should in incremented. maccurrent3=$(echo $arpout | grep -c $macdevice3) if [ $maccurrent3 -gt 0 ]; then maccurrent3=1 fi # Look for a change in status from the old known to the current status. # If it changed, update the ISY. Otherwise it leaves it as is. if [ $macconnected1 -ne $maccurrent1 ]; then if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Updating ISY upon change of Aaron's iPhone. It is now: "$maccurrent1; fi #Debug output to the syslog. wget -q http://$username:$password@$IPAddr/rest/vars/set/2/$isyvar1/$maccurrent1 fi if [ $macconnected2 -ne $maccurrent2 ]; then if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Updating ISY upon change of Rachel's iPhone. It is now: "$maccurrent2; fi #Debug output to the syslog. wget -q http://$username:$password@$IPAddr/rest/vars/set/2/$isyvar2/$maccurrent2 fi if [ $macconnected3 -ne $maccurrent3 ]; then if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Updating ISY upon change of Aaron's iPad Air. It is now: "$maccurrent3; fi #Debug output to the syslog. wget -q http://$username:$password@$IPAddr/rest/vars/set/2/$isyvar3/$maccurrent3 fi # Update the known status from the current. Ready for the next loop. macconnected1=$maccurrent1; macconnected2=$maccurrent2; macconnected3=$maccurrent3; # Total up the number of devices connected. let currentconnected=$macconnected1+$macconnected2+$macconnected3 #remember to add new devices to add up here in the total. # Look for a change, and update the ISY. if [ $connected -ne $currentconnected ]; then if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Updating ISY upon status change in the total number of devices. It is now: " $currentconnected; fi #Debug output to the syslog. wget -q http://$username:$password@$IPAddr/rest/vars/set/2/$isyvartot/$currentconnected fi connected=$currentconnected # Delay (sleep) depending on the connection status. # No devices connected could delay less. Once a device is connected, it could delay longer. if [ $connected -gt 0 ]; then sleep $delay_occupied else sleep $delay_unoccupied fi # For terminating the script only - uncomment to have the looping stop at X loops defined in variable: (internvalcounter X limit=**). #let counter=$counter+1 # Increment the interval counter let intervalcounter=$intervalcounter+1 # If the interval counter is greater than 10, reset it to 1. This done to output items to # the syslog once every 10 runs of the loop. The '10' below can be changed to output more # or less to the syslog as desired, or commented out entirely to stop the log output. if [ $intervalcounter -gt 2500 ]; then let intervalcounter=1 fi #end of testing loop done if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Loop limit of " $limit " reached. Script is stopping."; fi #Debug output to the syslog.
Steven M Castano Posted December 12, 2016 Author Posted December 12, 2016 Steven & ALL others wondering on the Proximity script for there routers. This script can be modified to work on DD-WRT and OpenWRT and ASUS Merlin firmware. Please change the below to fit your configuration. Here is the script that I am running on my ASUS RT-AC68U. I am running the latest version of the Merlin firmware designed for the router. See below. I spoke with the creator of the Merlin script and he gave me some advice on the script. Use "arpout=$(wl -i eth2 assoclist)" INSTREAD of the "arp -i br0". At least for the Merilin firmware as it found the devices even when they would fall asleep. I have further tweaking ahead since I'm moving away from MobiLinc for my locations, but this morning provided to be a success on my way to work. That's actually my script that I shared a long time ago! hahaha And yes, there have been some updates since then that allow for better checking on the ARP tables and such, but that should be enough to get you started. It's a general bash script, so any router firmware, OpenWRT, DD-WRT, etc... that you can run on a router, you'll be able to run this script. Mine even has some config options to change the arp commands and notification commands as needed.
memphis2k Posted December 12, 2016 Posted December 12, 2016 I started researching and there was a couple examples going back years. Like anything new, I have a few issues I have to solve. Double check the number of gps requests, and my iPad has left and arrived a couple times, (which hasn't moved since it's plugged in). Overall I'm happy and thanks again. Sunday was spent cleaning a carb on my 28" snowblower... that was a first but she runs perfect now. MN winters are fun!
mjpotts82 Posted May 17, 2017 Posted May 17, 2017 Haas there been any thought in packaging this all up into a Docker? I would think that would simplify installation and support a fair amount. Also, has anyone tried to use this script with inlfuxDB? I already have that running in a docker.
Steven M Castano Posted May 17, 2017 Author Posted May 17, 2017 Haas there been any thought in packaging this all up into a Docker? I would think that would simplify installation and support a fair amount. Also, has anyone tried to use this script with inlfuxDB? I already have that running in a docker. Sure, that could probably be done, but it's really just a python script, and I wrote it without many dependancies. Honestly, by the time you install the docker engine and everything else, you could just as easily just install mysql and python, in fact, most linux distributions will come with almost everything you need in python already installed. As for the influxDB, currently no it won't work. The python library used for the DB is specifically written for either MySQL or Postgres and the way the code has been written, the only driver and connection strings implemented are for MySQL. That being said, any MySQL compliant DB will work. I've personally testing MySQL 5.5, 5.6 and 5.7 as well as Percona and MariaDB and I run my production scripts on a MariaDB Galera cluster that I have installed already to support some other applications like WordPress, ownCloud and a few others. I have been thinking about writing an option into the script to stop the database storage of info anyway since all it's really used for is location history and some people aren't really interested in the history, just that the app is running in real time.
mjpotts82 Posted May 17, 2017 Posted May 17, 2017 I already have a few dockers running so I thought I would ask. I can spin up a linux VM, debian okay?
Steven M Castano Posted May 17, 2017 Author Posted May 17, 2017 Sure, it'll run on anything. It doesn't even have to be linux, that's just where most people run it. There's a few guys out there with it running on Debian, I run it on both Ubuntu and RedHat, and I even wrote in Windows detection to the do the paths for pid file storage and such properly. So the only dependency is that you have to have an OS that can run Python 2.7 and any MySQL compliant DB to connect to. I was running on a Raspberry Pi for a while and a buddy of mine runs 6 copies of it at the same time on a Raspberry Pi 2 along with MariaDB to do proximity for his entire family and it runs fine at less than 5% CPU.
glacket Posted July 28, 2017 Posted July 28, 2017 Sure, it'll run on anything. It doesn't even have to be linux, that's just where most people run it. There's a few guys out there with it running on Debian, I run it on both Ubuntu and RedHat, and I even wrote in Windows detection to the do the paths for pid file storage and such properly. So the only dependency is that you have to have an OS that can run Python 2.7 and any MySQL compliant DB to connect to. I was running on a Raspberry Pi for a while and a buddy of mine runs 6 copies of it at the same time on a Raspberry Pi 2 along with MariaDB to do proximity for his entire family and it runs fine at less than 5% CPU. Hi, I would like to run this script on a raspberry pi and I am not clear on what prerequisites are needed to run it. I want to use raspbian. Do I need to install and setup a mysql db on the pi for the script to call from? if so are there more detailed instructions on how to do that. I am not sure where to start.
glacket Posted July 31, 2017 Posted July 31, 2017 well i figured out some things. i loaded a pi with raspbian jessie, install mysql, php and arp-scan to pull arp data into a db. I got most of that done except i am having trouble getting the data from arp scan to populate in the db. aside from that what would i need to change in the script provided in this thread to pull from the db when running locally on the pi? anyone?
Steven M Castano Posted August 1, 2017 Author Posted August 1, 2017 Hi, I would like to run this script on a raspberry pi and I am not clear on what prerequisites are needed to run it. I want to use raspbian. Do I need to install and setup a mysql db on the pi for the script to call from? if so are there more detailed instructions on how to do that. I am not sure where to start. Which script are you trying to run... the router based shell script that does the WiFi Proximity checking, or the python script that does the iCloud API checking for device location?
glacket Posted August 1, 2017 Posted August 1, 2017 Hi Steven, I am trying to run the router based shell script that does proximity checking on my ubiquiti. I just found out yesterday that the pi at some point no longer see my phones arp when its asleep. my edgemax router does, so now I am trying to run the script on my ubiquiti edgemax pro, it runs on Linux. I would need to add it to cron.d to make it run at startup continuously, not sure if I have to change anything else.
glacket Posted August 2, 2017 Posted August 2, 2017 I forgot that some of the commands in these wifi proximity scripts are specific to the chipset. I found another version of the wifi proximity script above, on another thread that had been converted to unbuntu. That does not return any errors when launching, however there is not logging and the isy state variables do not change so something still does not appear to be working right.
mcgedoue Posted October 5, 2017 Posted October 5, 2017 Seems like latest iOS 11 requires 2 factor authentication now, is this working reliably? I’m considering using this script. Sent from my iPhone using Tapatalk
rccoleman Posted October 5, 2017 Posted October 5, 2017 I’m using Locative now, but 2FA was working fine for me with this script a while ago.
nathagt Posted October 19, 2017 Posted October 19, 2017 I am trying to get this working again after getting a new iPhone awhile back. I'm having issues running the listdevices.py command. I'm getting back all this. I've downloaded the latest code from github and upgraded to python 2.7.9. The requirements.txt install reports back that every requirement is already satisfied. Just to be safe I did a pip uninstall and then ran the pip install requirements again. Both were successful. Any help would be appreciated. iphonelocation.ini exists, continuing. Authenticatin to the iCloud API. Traceback (most recent call last): File "./listdevices.py", line 95, in <module> device_string = str(api.devices) File "C:\Python27\lib\site-packages\pyicloud\base.py", line 270, in devices self.params File "C:\Python27\lib\site-packages\pyicloud\services\findmyiphone.py", line 2 8, in __init__ self.refresh_client() File "C:\Python27\lib\site-packages\pyicloud\services\findmyiphone.py", line 4 4, in refresh_client 'selectedDevice': 'all', File "C:\Python27\lib\site-packages\requests\sessions.py", line 555, in post return self.request('POST', url, data=data, json=json, **kwargs) File "C:\Python27\lib\site-packages\pyicloud\base.py", line 67, in request response = super(PyiCloudSession, self).request(*args, **kwargs) File "C:\Python27\lib\site-packages\requests\sessions.py", line 508, in reques t resp = self.send(prep, **send_kwargs) File "C:\Python27\lib\site-packages\requests\sessions.py", line 618, in send r = adapter.send(request, **kwargs) File "C:\Python27\lib\site-packages\requests\adapters.py", line 506, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='p41-fmipweb.icloud.com', port=443): Max retries exceeded with url: /fmipservice/client/web/refreshClient ?clientBuildNumber=14E45&dsid=58102158&clientId=76D7750F-B4ED-11E7-8F58-38607705 FB73 (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certific ate verify failed (_ssl.c:581)'),)) The other issue which I haven't even begun to tackle is I need to use two-factor authentication. I don't see a setting for that anywhere in the iphonelocation.ini file. For now, I've disabled 2FA for my iCloud account to help resolve my first issue.
nathagt Posted October 19, 2017 Posted October 19, 2017 If anyone stumbles on this post or has this same issue, I resolved it by using the following: pip uninstall -y certifi pip install certifi==2015.04.28 I'm not sure if this is the best way to resolve it or not. If anyone has a better suggestion please chime in.
someguy Posted December 3, 2017 Posted December 3, 2017 I found what it surely a much simpler solution for proximity with some limitations: The mobile app "MobiLinc Pro" (which I use on my iphone) uses the location of the iphone to alert the ISY of proximity to any preset point. You can set as many points and a radius to each point and your ISY is (as far as I can tell) instantly alerted if you enter or leave this circular geofence. Here are the details: https://mobilinc.com/features/geofence/ Although it has limitations, it doesn't seem to have the concern of running the phone's battery down and most importantly: it is much simpler to set up than the proximity script that this thread is suggesting.
rccoleman Posted December 3, 2017 Posted December 3, 2017 (edited) That requires a substantial investment in Mobilinc and an additional purchase of the geofence module, while this is free. I do have Mobilinc, but chose not to rely on their geofence solution because I’ve generally found them to be unreliable. That was my original motivation for moving to this, but I’ve since transitioned to the free Locative app that has reliable geofence support and a more friendly interface. I think that the problem with most geofence apps is that they’re likely to crash when they trigger due to issues around network reliability, and this works around that by relying on Apple’s presumably hardened Find my iPhone service. I certainly see this problem with IFTTT geofences, and they almost never work properly for me. Maybe Mobilinc is more reliable, but this is a free alternative that takes a new and creative approach. Edited December 3, 2017 by rccoleman
dpower Posted January 4, 2018 Posted January 4, 2018 Steven... This may be an ignorant question but is there any reason your script wouldn't run on a Windows 7 box?
Steven M Castano Posted January 4, 2018 Author Posted January 4, 2018 Steven... This may be an ignorant question but is there any reason your script wouldn't run on a Windows 7 box? No, it should run without a problem... you would just have to make sure all the python dependencies get installed. I don't think I've tested it on windows myself, but it's written in standard python 2.7. As long as you can get python and pip to install, it should work. Are you getting any specific errors? Sent from my iPhone using Tapatalk
Recommended Posts