iOS / ISY Proximity Script

[Steven M Castano]

Steven:

 

as far as this step goes:

can you expound on how to do this using GitHub, HeidiSQL, MariaDB and Python?  I don't understand what you mean here.  thanks again, Steven.

 

With HeidieSQL you would connect to your database, so on your local host, and you can just create a blank database using the gui. Name it anything you'd like and use the name and the username and password you used in HeidiSQL to connect in the iphonelocation.ini file as your database setting. Your hostname would be either "localhost" or "127.0.0.1" then and the standard port of 3306 should be fine.

[someguy]

Okay, I have a database created in HeidiSQL.  What next?

 

[memphis2k]

Steven,

 

A while back in the thread, you talked about the script you use on your ASUS router.  I have hacked together a script in PHP that will "ping" devices and displays a result if the device is alive.  Whats your thoughts on something like this as opposed to the ARP script / Disconnect one you have.  Battery life etc.

 

I do have an ASUS AC1900 RT-AC68U Router configured as an Access-Point only.  I run SOPHOS firewall in a VM in a Server20212 R2.  I do have three RasPi's too, 1-3.  I have PHP and Python running on the Server too.  I'm willing to mod the Wifi router too...

 

Knowing what I use, just looking for the best approach on monitoring my devices when they are home and changing the values for this script.

 

Thanks for your .02

~Aaron

[Steven M Castano]

Okay, I have a database created in HeidiSQL.  What next?

 

Capture.JPG

Sorry, I vanished there for a bit... I've had a few crazy weeks.

 

Now that you've created the database, you just put the database name and the mysql login and password in the iphonelocation.ini file and the script will take care of the rest of creating the tables and stuff you need.

 

Then you can fill in your iCloud account instructions and keep following the instructions on how to list devices and select the one you want to use for GPS location.

[Steven M Castano]

Steven,

 

A while back in the thread, you talked about the script you use on your ASUS router.  I have hacked together a script in PHP that will "ping" devices and displays a result if the device is alive.  Whats your thoughts on something like this as opposed to the ARP script / Disconnect one you have.  Battery life etc.

 

I do have an ASUS AC1900 RT-AC68U Router configured as an Access-Point only.  I run SOPHOS firewall in a VM in a Server20212 R2.  I do have three RasPi's too, 1-3.  I have PHP and Python running on the Server too.  I'm willing to mod the Wifi router too...

 

Knowing what I use, just looking for the best approach on monitoring my devices when they are home and changing the values for this script.

 

Thanks for your .02

~Aaron

 

I tried doing this for a while myself and found there's 2 major problems:

 

1) The ping response is just as flaky as the device staying on wifi. It won't respond to pings a lot of the times even when it's there... so you're still stuck with the same problem with the ARP method in that you have to account for temporary disconnects

 

2) Since a ping is an active part of the TCP/IP stack, the phone itself has to create the ping response and send data back. So you're actually asking the phone to do work. Work = battery = does dead faster. The ARP method is totally passive and really doesn't depend on the iOS device at all, so you still have problem number 1, but you don't even negatively effect battery life.

 

I would still point you  in the direction of a passive monitor. If you've got any way of writing a script on the firewall or mod'ing the router so you can run a bash script, then those are probably your best bets. You just have to make sure you're monitoring things from a location that ALL wifi traffic has to pass through. If you only have the one ASUS router as your only WiFi access point, that might be the perfect place to do it. Run OpenWRT on it and I can clean up and post the bash script I use to look at the ARP table.

[memphis2k]

I tried doing this for a while myself and found there's 2 major problems:

 

1) The ping response is just as flaky as the device staying on wifi. It won't respond to pings a lot of the times even when it's there... so you're still stuck with the same problem with the ARP method in that you have to account for temporary disconnects

 

2) Since a ping is an active part of the TCP/IP stack, the phone itself has to create the ping response and send data back. So you're actually asking the phone to do work. Work = battery = does dead faster. The ARP method is totally passive and really doesn't depend on the iOS device at all, so you still have problem number 1, but you don't even negatively effect battery life.

 

I would still point you  in the direction of a passive monitor. If you've got any way of writing a script on the firewall or mod'ing the router so you can run a bash script, then those are probably your best bets. You just have to make sure you're monitoring things from a location that ALL wifi traffic has to pass through. If you only have the one ASUS router as your only WiFi access point, that might be the perfect place to do it. Run OpenWRT on it and I can clean up and post the bash script I use to look at the ARP table.

 

 

Steven & ALL others wondering on the Proximity script for there routers.  This script can be modified to work on DD-WRT and OpenWRT and ASUS Merlin firmware.  Please change the below to fit your configuration.

 

Here is the script that I am running on my ASUS RT-AC68U.  I am running the latest version of the Merlin firmware designed for the router.  See below.  I spoke with the creator of the Merlin script and he gave me some advice on the script.  Use "arpout=$(wl -i eth2 assoclist)" INSTREAD of the "arp -i br0".  At least for the Merilin firmware as it found the devices even when they would fall asleep. 

 

I have further tweaking ahead since I'm moving away from MobiLinc for my locations, but this morning provided to be a success on my way to work.

#!/bin/sh
######################################################################################################################
# Proximity detection                                                                                                #
#                                                                                                                    #
# A script designed to run on a router running DD-WRT to detect certain devices connected to the router.             #
# It runs at startup and runs continually, checking for a specific list of devices (phones/laptop, etc)              #
# that connect wirelessly to the router.  Once a device is connected, the status is updated on the ISY               #
# with either a 0=disconnected, or a 1=connected.  State variables are used on the ISY so that a change              #
# in status (0/1) can trigger a programe on the ISY.                                                                 #
#                                                                                                                    #
# The searching frequency can be adjusted to be slower/faster depending on your requirements. Searching too fast     #
# could burden your router.  Too slow might not update the status as necessary for the application.                  #
######################################################################################################################

# Make changes below
# This variable turns on or off notifications to the syslog for debugging.
# Set 'notifications' to '1' to enable logging, set it to '0' to turn them off.
notifications=1    # Logging is enable by default. Change this to turn it off.

# MAC address of each device to watch. Don't leave blank. All CAPS on MAC Letters

if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Setting MACs"; fi    #Debug output to the syslog.
macdevice1="****"    #Aarons iPhone
macdevice2="****"    #Rachels iPhone
macdevice3="**:**:**:**"    #Aarons iPad

if [ $notifications -eq 1 ]; then                #Debug output to the syslog.
# logger -t PX_DETECT "Set MACs:"                #Debug output to the syslog.
logger -t PX_DETECT "macdevice1=" $macdevice1    #Debug output to the syslog.
logger -t PX_DETECT "macdevice2=" $macdevice2    #Debug output to the syslog.
logger -t PX_DETECT "macdevice3=" $macdevice3    #Debug output to the syslog.
fi

# if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Setting ISY username, password and local IP"; fi  #Debug output to the syslog.
# ISY username, password, and IP Address
username="admin"
password="****"
IPAddr="192.168.1.***:***"
if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Set ISY username, password and local IP"; fi  #Debug output to the syslog.

# State variable numbers on the ISY corresponding to each MAC above
# isyvartot is the cummulative number of devices (of the 4 above) that are
# currently connected.
isyvar1=12
isyvar2=19  #CHANGE TO 5 ONCE THE OTHER SCRIPT SUPPORTS MULTIPLE USERS
isyvar3=43
isyvartot=22

# Occupied and unoccupied delay in seconds to check status
# Adjust for shorter/longer wait times.  For instance, when one device is already
# connected, you might want to check less frequently.  This could also delay the
# notification of a disconnect.
delay_occupied=4
delay_unoccupied=2

# initial testing loop count - uncomment the counter near the bottom of the script for testing only.
limit=5
# if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Testing loop limit: " $limit; fi    #Debug output to the syslog.

###############################################
# do not change below here
###############################################

# DO NOT CHANGE THE LINE BELOW. THE "XXXX" IS WHAT *SHOULD& BE THERE!!!
# This creates a variable use to store the output from a single arp command
# so it can be tested multiple times for the presence of any of your devices.
# It is used to lower the number of times the arp command is run to a single
# time per test cycle. It also store the output in RAM as opposed to using
# additional resources to write it out to disk/nvram.
arpout=XXXX

sleep
#initialize internal variables

if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Setting connection status to initial state of -1"; fi #Debug output to the syslog.
# status of each MAC. 0=disconnected. 1=connected.  -1 initially forces isy update first loop
macconnected1=-1
macconnected2=-1
macconnected3=-1
connected=-1
# total number of currently conencted devices.
currentconnected=0

# initializing the counter at 1
counter=1

# initializing the interval counter to be used to output less frequently
# to the syslog service during the loop.
intervalcounter=1

if [ $notifications -eq 1 ]; then logger -t PX_DETECT "All variables and controls are set, beginning script logic/testing loop"; fi    #Debug output to the syslog.

# Initial testing loop.  Will run continually after testing is complete
while [ $counter -lt $limit ]; do

#if the interval counter is 1 then make a log entry.  Uncomment to show Interval loop count status

# if [ $notifications -eq 1 ]; then
#  if [ $intervalcounter -eq 1 ]; then
#     logger -t PX_DETECT "Test loop " $counter " of " $limit "is running."
#  fi
# fi

# Reset current status. Two variables are used for each device.  The past known status and the current
# status.  Only a change is reported to the ISY.  Otherwise, it would constantly be updating the ISY with
# the current status creating unnecessary traffic for both the router and the ISY
maccurrent1=0;
maccurrent2=0;
maccurrent3=0;

# This section compares the list of MACs you entered above against the output of a
# single arp command listing the MACs associated with the wired bridge of your
# router. Interface "br0"

# This command runs arp once and saves it in the 'arpout' variable in memory - Old command is (arp -i br0)
arpout=$(wl -i eth2 assoclist)

# The following runs 'grep' against the output form 'arp' in order to see if your
# first entered MAC address is present on the network.
maccurrent1=$(echo $arpout | grep -c $macdevice1)
if [ $maccurrent1 -gt 0 ]; then
   maccurrent1=1
fi

# The following section checks the second MAC identical to method discussed in the
# section directly above. To check additional MACs this section can be copied and
# the numbers following 'maccurrent' and 'macdevice' should in incremented.
maccurrent2=$(echo $arpout | grep -c $macdevice2)
if [ $maccurrent2 -gt 0 ]; then
   maccurrent2=1
fi

# The following section checks the third MAC identical to method discussed in the
# section directly above. To check additional MACs this section can be copied and
# the numbers following 'maccurrent' and 'macdevice' should in incremented.
maccurrent3=$(echo $arpout | grep -c $macdevice3)
if [ $maccurrent3 -gt 0 ]; then
   maccurrent3=1
fi

# Look for a change in status from the old known to the current status.
# If it changed, update the ISY. Otherwise it leaves it as is.

if [ $macconnected1 -ne $maccurrent1 ]; then
   if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Updating ISY upon change of Aaron's iPhone.  It is now: "$maccurrent1; fi   #Debug output to the syslog.
   wget -q http://$username:$password@$IPAddr/rest/vars/set/2/$isyvar1/$maccurrent1
fi

if [ $macconnected2 -ne $maccurrent2 ]; then
   if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Updating ISY upon change of Rachel's iPhone. It is now: "$maccurrent2; fi   #Debug output to the syslog.
   wget -q http://$username:$password@$IPAddr/rest/vars/set/2/$isyvar2/$maccurrent2
fi

if [ $macconnected3 -ne $maccurrent3 ]; then
   if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Updating ISY upon change of Aaron's iPad Air. It is now: "$maccurrent3; fi   #Debug output to the syslog.
   wget -q http://$username:$password@$IPAddr/rest/vars/set/2/$isyvar3/$maccurrent3
fi

# Update the known status from the current.  Ready for the next loop.
macconnected1=$maccurrent1;
macconnected2=$maccurrent2;
macconnected3=$maccurrent3;

# Total up the number of devices connected.
let currentconnected=$macconnected1+$macconnected2+$macconnected3 #remember to add new devices to add up here in the total.

# Look for a change, and update the ISY.
if [ $connected -ne $currentconnected ]; then
   if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Updating ISY upon status change in the total number of devices. It is now: " $currentconnected; fi #Debug output to the syslog.
   wget -q http://$username:$password@$IPAddr/rest/vars/set/2/$isyvartot/$currentconnected
fi

connected=$currentconnected

# Delay (sleep) depending on the connection status.
# No devices connected could delay less.  Once a device is connected, it could delay longer.

if [ $connected -gt 0 ]; then
   sleep $delay_occupied
else
   sleep $delay_unoccupied
fi

# For terminating the script only - uncomment to have the looping stop at X loops defined in variable: (internvalcounter X limit=**).
#let counter=$counter+1

# Increment the interval counter
let intervalcounter=$intervalcounter+1

# If the interval counter is greater than 10, reset it to 1. This done to output items to
# the syslog once every 10 runs of the loop. The '10' below can be changed to output more
# or less to the syslog as desired, or commented out entirely to stop the log output.
if [ $intervalcounter -gt 2500 ]; then
   let intervalcounter=1
fi

#end of testing loop
done

if [ $notifications -eq 1 ]; then logger -t PX_DETECT "Loop limit of " $limit " reached. Script is stopping."; fi   #Debug output to the syslog.

[Steven M Castano]

 

Steven & ALL others wondering on the Proximity script for there routers.  This script can be modified to work on DD-WRT and OpenWRT and ASUS Merlin firmware.  Please change the below to fit your configuration.

 

Here is the script that I am running on my ASUS RT-AC68U.  I am running the latest version of the Merlin firmware designed for the router.  See below.  I spoke with the creator of the Merlin script and he gave me some advice on the script.  Use "arpout=$(wl -i eth2 assoclist)" INSTREAD of the "arp -i br0".  At least for the Merilin firmware as it found the devices even when they would fall asleep. 

 

I have further tweaking ahead since I'm moving away from MobiLinc for my locations, but this morning provided to be a success on my way to work. 

 

That's actually my script that I shared a long time ago! hahaha

 

And yes, there have been some updates since then that allow for better checking on the ARP tables and such, but that should be enough to get you started. It's a general bash script, so any router firmware, OpenWRT, DD-WRT, etc... that you can run on a router, you'll be able to run this script. Mine even has some config options to change the arp commands and notification commands as needed.

[memphis2k]

I started researching and there was a couple examples going back years. Like anything new, I have a few issues I have to solve. Double check the number of gps requests, and my iPad has left and arrived a couple times, (which hasn't moved since it's plugged in). Overall I'm happy and thanks again.

 

Sunday was spent cleaning a carb on my 28" snowblower... that was a first but she runs perfect now. MN winters are fun!

[mjpotts82]

Haas there been any thought in packaging this all up into a Docker? I would think that would simplify installation and support a fair amount.

 

Also, has anyone tried to use this script with inlfuxDB? I already have that running in a docker.

[Steven M Castano]

Haas there been any thought in packaging this all up into a Docker? I would think that would simplify installation and support a fair amount.

 

Also, has anyone tried to use this script with inlfuxDB? I already have that running in a docker.

 

Sure, that could probably be done, but it's really just a python script, and I wrote it without many dependancies. Honestly, by the time you install the docker engine and everything else, you could just as easily just install mysql and python, in fact, most linux distributions will come with almost everything you need in python already installed.

 

As for the influxDB, currently no it won't work. The python library used for the DB is specifically written for either MySQL or Postgres and the way the code has been written, the only driver and connection strings implemented are for MySQL. That being said, any MySQL compliant DB will work. I've personally testing MySQL 5.5, 5.6 and 5.7 as well as Percona and MariaDB and I run my production scripts on a MariaDB Galera cluster that I have installed already to support some other applications like WordPress, ownCloud and a few others.

 

I have been thinking about writing an option into the script to stop the database storage of info anyway since all it's really used for is location history and some people aren't really interested in the history, just that the app is running in real time.

[mjpotts82]

I already have a few dockers running so I thought I would ask. I can spin up a linux VM, debian okay?

[Steven M Castano]

Sure, it'll run on anything. It doesn't even have to be linux, that's just where most people run it. There's a few guys out there with it running on Debian, I run it on both Ubuntu and RedHat, and I even wrote in Windows detection to the do the paths for pid file storage and such properly. So the only dependency is that you have to have an OS that can run Python 2.7 and any MySQL compliant DB to connect to.

 

I was running on a Raspberry Pi for a while and a buddy of mine runs 6 copies of it at the same time on a Raspberry Pi 2 along with MariaDB to do proximity for his entire family and it runs fine at less than 5% CPU.

[glacket]

Sure, it'll run on anything. It doesn't even have to be linux, that's just where most people run it. There's a few guys out there with it running on Debian, I run it on both Ubuntu and RedHat, and I even wrote in Windows detection to the do the paths for pid file storage and such properly. So the only dependency is that you have to have an OS that can run Python 2.7 and any MySQL compliant DB to connect to.

 

I was running on a Raspberry Pi for a while and a buddy of mine runs 6 copies of it at the same time on a Raspberry Pi 2 along with MariaDB to do proximity for his entire family and it runs fine at less than 5% CPU.

Hi, I would like to run this script on a raspberry pi and I am not clear on what prerequisites are needed to run it. I want to use raspbian. Do I need to install and setup a mysql db on the pi for the script to call from? if so are there more detailed instructions on how to do that. I am not sure where to start.

[glacket]

well i figured out some things. i loaded a pi with raspbian jessie, install mysql, php and arp-scan to pull arp data into a db. I got most of that done except i am having trouble getting the data from arp scan to populate in the db. aside from that what would i need to change in the script provided in this thread to pull from the db when running locally on the pi? anyone?

[Steven M Castano]

Hi, I would like to run this script on a raspberry pi and I am not clear on what prerequisites are needed to run it. I want to use raspbian. Do I need to install and setup a mysql db on the pi for the script to call from? if so are there more detailed instructions on how to do that. I am not sure where to start.

 

Which script are you trying to run... the router based shell script that does the WiFi Proximity checking, or the python script that does the iCloud API checking for device location?

[glacket]

Hi Steven,

 

I am trying to run the router based shell script that does proximity checking on my ubiquiti. I just found out yesterday that the pi at some point no longer see my phones arp when its asleep. my edgemax router does, so now I am trying to run the script on my ubiquiti edgemax pro, it runs on Linux. I would need to add it to cron.d to make it run at startup continuously, not sure if I have to change anything else.

[glacket]

I forgot that some of the commands in these wifi proximity scripts are specific to the chipset. I found another version of the wifi proximity script above, on another thread that had been converted to unbuntu. That does not return any errors when launching, however there is not logging and the isy state variables do not change so something still does not appear to be working right.

[mcgedoue]

Seems like latest iOS 11 requires 2 factor authentication now, is this working reliably? I’m considering using this script.

 

 

Sent from my iPhone using Tapatalk

[rccoleman]

I’m using Locative now, but 2FA was working fine for me with this script a while ago.

[nathagt]

I am trying to get this working again after getting a new iPhone awhile back. I'm having issues running the listdevices.py command. I'm getting back all this. I've downloaded the latest code from github and upgraded to python 2.7.9. The requirements.txt install reports back that every requirement is already satisfied. Just to be safe I did a pip uninstall and then ran the pip install requirements again. Both were successful. Any help would be appreciated. 

iphonelocation.ini exists, continuing.
Authenticatin to the iCloud API.
Traceback (most recent call last):
  File "./listdevices.py", line 95, in <module>
    device_string = str(api.devices)
  File "C:\Python27\lib\site-packages\pyicloud\base.py", line 270, in devices
    self.params
  File "C:\Python27\lib\site-packages\pyicloud\services\findmyiphone.py", line 2
8, in __init__
    self.refresh_client()
  File "C:\Python27\lib\site-packages\pyicloud\services\findmyiphone.py", line 4
4, in refresh_client
    'selectedDevice': 'all',
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 555, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "C:\Python27\lib\site-packages\pyicloud\base.py", line 67, in request
    response = super(PyiCloudSession, self).request(*args, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 508, in reques
t
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 506, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='p41-fmipweb.icloud.com',
 port=443): Max retries exceeded with url: /fmipservice/client/web/refreshClient
?clientBuildNumber=14E45&dsid=58102158&clientId=76D7750F-B4ED-11E7-8F58-38607705
FB73 (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certific
ate verify failed (_ssl.c:581)'),))

The other issue which I haven't even begun to tackle is I need to use two-factor authentication. I don't see a setting for that anywhere in the iphonelocation.ini file. For now, I've disabled 2FA for my iCloud account to help resolve my first issue. 

[nathagt]

If anyone stumbles on this post or has this same issue, I resolved it by using the following:

pip uninstall -y certifi
pip install certifi==2015.04.28

I'm not sure if this is the best way to resolve it or not. If anyone has a better suggestion please chime in. 

[someguy]

I found what it surely a much simpler solution for proximity with some limitations:

 

The mobile app "MobiLinc Pro" (which I use on my iphone) uses the location of the iphone to alert the ISY of proximity to any preset point.  You can set as many points and a radius to each point and your ISY is (as far as I can tell) instantly alerted if you enter or leave this circular geofence.  Here are the details:

 

https://mobilinc.com/features/geofence/

 

Although it has limitations, it doesn't seem to have the concern of running the phone's battery down and most importantly:  it is much simpler to set up than the proximity script that this thread is suggesting.  

[rccoleman]

That requires a substantial investment in Mobilinc and an additional purchase of the geofence module, while this is free. I do have Mobilinc, but chose not to rely on their geofence solution because I’ve generally found them to be unreliable. That was my original motivation for moving to this, but I’ve since transitioned to the free Locative app that has reliable geofence support and a more friendly interface.

 

I think that the problem with most geofence apps is that they’re likely to crash when they trigger due to issues around network reliability, and this works around that by relying on Apple’s presumably hardened Find my iPhone service. I certainly see this problem with IFTTT geofences, and they almost never work properly for me. Maybe Mobilinc is more reliable, but this is a free alternative that takes a new and creative approach.

[dpower]

Steven... This may be an ignorant question but is there any reason your script wouldn't run on a Windows 7 box?

[Steven M Castano]

Steven... This may be an ignorant question but is there any reason your script wouldn't run on a Windows 7 box?

No, it should run without a problem... you would just have to make sure all the python dependencies get installed. I don't think I've tested it on windows myself, but it's written in standard python 2.7. As long as you can get python and pip to install, it should work.

 

Are you getting any specific errors?

 

 

Sent from my iPhone using Tapatalk