I’ve been playing around with the ESP32 S2 Mini recently and wanted to try and create a device that allows you to track it’s location using only WiFi networks.
For a high-level overview of how this works. I created a python script to make the ESP32 connect to all open WiFi networks it finds. Once connected, it sends a DNS request to my server, which includes the BSSID of the access point it’s connected to. A script on the server then performs a lookup on wigle.net for the BSSID’s coordinates and generates a map showing the movement of the ESP32.
To describe how I did this in more detail I’m splitting this post into 4 headings. The first describes the hardware used. This is quite short as it consists of all off-the-shelf components. I then describe the script that runs on the ESP32. Then the configuration of the server which receives the DNS requests. Then finally the script used to generate a map showing the location of the ESP32.
The Hardware
The device consists of 3 components. The first is an ESP32 S2 D1 Mini. This has Circuit python installed to allow the device to run python scripts.
The second component is a battery shield.
This is soldered onto the top of the ESP32 and allows for the third component to be connected, a 3.7v 2000mAh Li-Po battery to power the ESP32.
Depending on your requirements this battery could be changed to a different form factor if you prefer the overall device to be smaller or have a larger capacity to run for longer. I chose this size because it’s just slightly bigger than the footprint of the ESP32.
These are connected together to create the tracking hardware.
ESP32 Software
As I explained in the introduction the ESP32 is running Circuit Python. I won’t go through the process of how I installed that here. There are many tutorials online.
To conserve battery life on the ESP32 i wanted to do as much processing as I could on the server rather than the ESP32. The aim for the ESP32 is to simply do the following:
Scan for WiFi networks and identify which ones are open.
1
2
3
4
5
openNetworks = []
for network in wifi.radio.start_scanning_networks():
if network.authmode[0] == wifi.AuthMode.OPEN:
openNetworks.append([network.ssid, network.bssid])
wifi.radio.stop_scanning_networks()
This performs a scan for all WiFi networks. It then iterates through each of them and checks if the authentication mode is Open. If the network is open it adds the SSID and the BSSID of the network to a list called openNetworks.
Connect to the open WiFi network.
Now we have a list of open WiFi networks in the vicinity of the ESP32. The next step is to connect to the WiFi network.
1
2
for network in openNetworks:
wifi.radio.connect(ssid=network[0], timeout=5.0)
As you can see a timeout is set to 5 seconds. If the ESP32 is moving quickly and becomes out of range of the open network, or other factors prevent the device from connecting this timeout ensures the script doesn’t hang when trying to connect.
Send a DNS request to my server
Within the same for-loop, once the ESP32 has successfully connected to the WiFi network it then sends a DNS request to my server.
1
url = "http://" + trackerName + "MACADDR" + str(hexlify(network[1]))[2:-1] + ".ns1.mydomain.co.uk"
This code first generates a URL to send the DNS request to. The first part of the URL is the trackerName. This is a global variable that can be unique for each ESP32. This allows for multiple ESP32 devices to be processed individually by the server. The second part of the URL is the string “MACADDR”. This allows the server to identify which requests are from an ESP32 and which are from general internet noise and can be ignored. Next is the BSSID for the AP the ESP32 has connected to. This is encoded in hex so it’s readable. The last part of the URL is the domain name for the server. In this code snippet, I’ve used ns1.mydomain.co.uk.
An example URL would look something like this:
http://tracker1MACADDRF310B83D7210.ns1.mydomain.co.uk
I chose to use DNS to exfiltrate the data to my server because many open Wifi networks block outgoing traffic unless you authenticate through a captive portal or something similar. Although this is the case DNS requests aren’t always blocked and so have more of a chance to circumvent this.
1
2
3
4
5
6
7
pool = socketpool.SocketPool(wifi.radio)
try:
pool.getaddrinfo(url, 80)
except Exception as e:
print(e)
wifi.radio.enabled=False
wifi.radio.enabled=True
This URL is stored in a variable called url. the getaddrinfo function is then run against this URL to perform a DNS request. This request is ultimately received by my server which I detail later on. Once this request is sent, the radio on the ESP32 is reset which disconnects it from the WiFi network. It then proceeds to the next open Wifi network in the openNetworks list. Once the ESP32 connects to all open networks, it rescans for more networks, and the whole process repeats until the battery depletes.
The following is the complete ESP32 script:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import wifi
import socketpool
from adafruit_binascii import hexlify
trackerName = "TestTracker"
openNetworks = []
for network in wifi.radio.start_scanning_networks():
if network.authmode[0] == wifi.AuthMode.OPEN:
openNetworks.append([network.ssid, network.bssid])
wifi.radio.stop_scanning_networks()
print(openNetworks)
for network in openNetworks:
print("joining network..." + network[0])
wifi.radio.connect(ssid=network[0], timeout=5.0)
print("my IP addr:", wifi.radio.ipv4_address)
url = "http://" + trackerName + "MACADDR" + str(hexlify(network[1]))[2:-1] + ".ns1.mydomain.co.uk"
print("Sending DNS request to: " + url)
pool = socketpool.SocketPool(wifi.radio)
try:
pool.getaddrinfo(url, 80)
except Exception as e:
print(e)
wifi.radio.enabled=False
wifi.radio.enabled=True
DNS Server Configuration
So we now have the hardware put together, and the ESP32 sends DNS requests to the server. The next stage is to configure the server to receive and process these requests.
I started by creating a VPS on Oracle cloud as they offer a free tier that works perfectly for this kind of project. However using something like Amazon EC2, Azure or self-hosting would all work just as well.
You next need to purchase a domain name. This is what the ESP32 performs the DNS request to. This needs to be configured so your server acts as a nameserver for that domain. In my case, I purchased a domain name through Amazon Route 53. I then created the following 2 records on that domain.
An A record with the name ns1.mydomain.co.uk pointing to the IP address of my server.
An NS record pointing to ns1.mydomain.co.uk.
This now means when a DNS request is made to blah.ns1.mydomain.co.uk, it is forwarded to my server to be processed.
I next set up DNS on my server. This can either be done manually using something link Bind9. Or automatically using OOB-Server. I recommend using OOB-Server it makes the process much easier.
Now the server has been configured it can be tested by browsing to the domain and viewing the named.log file to see if the request has been captured properly.
1
2
root@mydomain:/home/ubuntu# tail -f /var/log/named/named.log
29-Dec-2022 16:44:58.898 queries: info: client @0xffff600051d8 172.70.84.133#26462 (blab.ns1.mydomain.co.uk): query: blab.ns1.mydomain.co.uk IN A -E(0)D (10.0.0.60)
As you can see in the output above. the DNS query for blab.ns1.mydomain.co.uk was received. This won’t resolve to anything but that doesn’t matter. All we need is the request to be received by the server.
There are a couple of additional configuration changes I made. The first is to change the way the DNS server logs the requests, by default the log files are rotated once they reach 5mb in size which we don’t want. The map we generate later on uses the log file. So having all the requests in a single file is required. Do this by editing /etc/bind/named.conf.log and changing the size to something like 5 GB. This will prevent the log file rotation from occurring.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
logging {
channel bind_log {
file "/var/log/named/named.log" versions 3 size 5g;
severity info;
print-category yes;
print-severity yes;
print-time yes;
};
category default { bind_log; };
category update { bind_log; };
category update-security { bind_log; };
category security { bind_log; };
category queries { bind_log; };
category lame-servers { null; };
};
If your server also uses logratate this needs to be altered as well. This rotates the log files after X amount of days. What you alter this to can vary depending on how you would like the devices to be tracked. This is configured using /etc/logrotate/bind.
1
2
3
4
5
6
7
8
9
10
11
12
/var/log/named/named.log {
daily
missingok
rotate 7
compress
delaycompress
notifempty
create 644 bind bind
postrotate
/usr/sbin/invoke-rc.d bind9 reload > /dev/null
endscript
}
As you can see the DNS log file is rotated every 7 days. This means that when you generate the map it will only show the location for the past 7 days. Essentially this means it will show all locations for the ESP32 as it won’t run for 7 days straight off the battery.
Now this has been set up, I turned on the ESP32 and had a walk around. As I walk around the ESP32 connects to open WiFi networks and sends DNS requests to my server containing the BSSID. The following is an example of a single request from the named.log file:
27-Nov-2022 11:12:56.347 queries: info: client @0xffff70007ec8 3.9.41.14#11115 (testtrackermacaddr44d9e7ade560.ns1.mydomain.co.uk): query: testtrackermacaddr44d9e7ade560.ns1.mydomain.co.uk IN A -E(0)DC (10.0.0.60)
Map Generation
Now we have a log file containing all the BSSID’s the ESP32 has connected to. The final stage is to create a map plotting the location. To do this I use the wigle.net API. Wigle is a site that plots many wireless networks on a map. Their API allows you to look up a BSSID and it returns the latitude and longitude for that access point.
A function is first created called wifleLookup which takes the mac address and performs a lookup using the wigle API to get the latitude and longitude for the access point.
1
2
3
4
5
def wigleLookup(theMacAddr):
payload = {'netid': theMacAddr, 'api_key': (wigle_username + wigle_password).encode()} # FETCHING JSON RESPONSE FROM WiGLE
response = requests.get(url='https://api.wigle.net/api/v2/network/search', params=payload, auth=HTTPBasicAuth(wigle_username, wigle_password)).json() # PRINTING TO CHECK RESPONSE:
latlong = [str(response['results'][0]['trilat']), str(response['results'][0]['trilong'])]
return latlong
The script then opens the named.log file which contains all the DNS requests described previously.
1
2
3
4
5
6
7
8
9
10
logfile = open("/var/log/named/named.log", "r")
records = []
for line in logfile:
if "MACADDR" in line:
datetime = line.split(' ')[0] + " " + line.split(' ')[1]
query = line.split(' ')[7]
trackerName = query[8:query.index("MACADDR")]
macAddr = query[query.index("MACADDR") + 7: query.index(".")]
macAddr = ':'.join(macAddr[i:i+2] for i in range(0, len(macAddr), 2))
We iterate through each line of the log file and check to see if the string MACADDR is present in the line. If it is then we know that it is a request from the ESP32. Using a combination of splits and slices the date/time, tracker name, and BSSID are all pulled from the request and stored in the respective variable. This wingleLookup function is then run against the request:
1
2
3
4
5
6
try:
latlong = wigleLookup(macAddr)
print("At " + datetime + " The Tracker " + trackerName + " set a query with the MAC address " + macAddr + " at " + latlong[0] + " " + latlong[1])
records.append([datetime, trackerName, macAddr, latlong[0], latlong[1]])
except(IndexError):
pass
As you can see this appends the date/time, tracker name, mac address, and coordinates to a list called records. If the lookup fails for any reason it simply continues to the next DNS query.
We next iterate through the records list to generate the map:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
if len(records) > 0:
loc = []
m = folium.Map(location=[records[0][3], records[0][4]], zoom_start=15)
#Places the starting marker
folium.Marker([records[0][3], records[0][4]], icon=DivIcon(icon_size=(30,30), icon_anchor=(80,20), html=f'<div style="font-size: 14pt; text-shadow: -1px -1px 0 #000, 1px -1px 0 #000, -1px 1px 0 #000, 1px 1px 0 #000; color: white;">Start</div>')).add_to(m)
#places the finish marker
folium.Marker([records[-1][3], records[-1][4]], icon=DivIcon(icon_size=(30,30), icon_anchor=(80,20), html=f'<div style="font-size: 14pt; text-shadow: -1px -1px 0 #000, 1px -1px 0 #000, -1px 1px 0 #000, 1px 1px 0 #000; color: white;">Finish</div>')).add_to(m)
#the FOR loop adds the markers for the different records on the map.
for record in records:
loc.append((float(record[3]), float(record[4])))
folium.Marker(location=(float(record[3]), float(record[4])),
radius=10,
color='red',
fill_color='red',
fill_opacity=0.7,
tooltip="Date / Time: " + record[0] + "<br>" + "Mac Address: " + record[2]
).add_to(m)
folium.PolyLine(loc, color='blue', weight=15, opacity=0.8).add_to(m)
m.save("index.html")
This map is created with a start and finish marker so you can identify the direction of the ESP32. It then adds an individual marker for each WiFi network it connects to. You can mouse over each marker to see the date/time the ESP32 connected to that access point.
The following is the complete script for the map generation:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
import requests
from requests.auth import HTTPBasicAuth
import folium
from folium.features import DivIcon
# SETTING WiGLE USERNAME & PASSWORD FOR API CALL:
wigle_username = 'ENTER USERNAME HERE'
wigle_password = 'ENTER PASSWORD HERE'# SETTING PARAMETERS:
# Performs a lookup on Wigle returning the coordinates for the BSSID.
def wigleLookup(theMacAddr):
payload = {'netid': theMacAddr, 'api_key': (wigle_username + wigle_password).encode()} # FETCHING JSON RESPONSE FROM WiGLE
response = requests.get(url='https://api.wigle.net/api/v2/network/search', params=payload, auth=HTTPBasicAuth(wigle_username, wigle_password)).json() # PRINTING TO CHECK RESPONSE:
latlong = [str(response['results'][0]['trilat']), str(response['results'][0]['trilong'])]
return latlong
logfile = open("/var/log/named/named.log", "r")
# Iterates through the DNS log file. pulls the date/time, tracker name and BSSID from the request and performs a lookup for the coordinates. These are saved in a list called records.
records = []
for line in logfile:
if "MACADDR" in line:
datetime = line.split(' ')[0] + " " + line.split(' ')[1]
query = line.split(' ')[7]
trackerName = query[8:query.index("MACADDR")]
macAddr = query[query.index("MACADDR") + 7: query.index(".")]
macAddr = ':'.join(macAddr[i:i+2] for i in range(0, len(macAddr), 2))
try:
latlong = wigleLookup(macAddr)
print("At " + datetime + " The Tracker " + trackerName + " set a query with the MAC address " + macAddr + " at " + latlong[0] + " " + latlong[1])
records.append([datetime, trackerName, macAddr, latlong[0], latlong[1]])
except(IndexError):
pass
# Iterates through the list of coordinates, generates a map from them which is saved in the same folder called index.html.
if len(records) > 0:
loc = []
m = folium.Map(location=[records[0][3], records[0][4]], zoom_start=15)
#Places the starting marker
folium.Marker([records[0][3], records[0][4]], icon=DivIcon(icon_size=(30,30), icon_anchor=(80,20), html=f'<div style="font-size: 14pt; text-shadow: -1px -1px 0 #000, 1px -1px 0 #000, -1px 1px 0 #000, 1px 1px 0 #000; color: white;">Start</div>')).add_to(m)
#places the finish marker
folium.Marker([records[-1][3], records[-1][4]], icon=DivIcon(icon_size=(30,30), icon_anchor=(80,20), html=f'<div style="font-size: 14pt; text-shadow: -1px -1px 0 #000, 1px -1px 0 #000, -1px 1px 0 #000, 1px 1px 0 #000; color: white;">Finish</div>')).add_to(m)
#the FOR loop adds the markers for the different records on the map.
for record in records:
loc.append((float(record[3]), float(record[4])))
folium.Marker(location=(float(record[3]), float(record[4])),
radius=10,
color='red',
fill_color='red',
fill_opacity=0.7,
tooltip="Date / Time: " + record[0] + "<br>" + "Mac Address: " + record[2]
).add_to(m)
folium.PolyLine(loc, color='blue', weight=15, opacity=0.8).add_to(m)
m.save("index.html")
Conclusion
In general, it all works quite well. There are a number of enhancements that could be made to increase its efficiency. For example, preventing the device from sending multiple DNS requests from a single AP while stationary. Maybe limiting it to one every hour for example. Its accuracy is also proportional to the number of open WiFi networks in its vicinity. The map below shows the blue line generated by the ESP32. The green line shows my actual walking route.
As you can see when walking through the city center there were a lot more markers present, so the blue line and green line matched up nicely. This is because there are a lot of open WiFi networks in the city center. Many of the stores have free open WiFi. However, when walking through the residential areas the open WiFi networks are much less prevalent, so the accuracy is poorer.
If anyone decides to build on this project at all please let me know. I would be interested to see where it leads. If you have any questions I can be contacted via email at j@meswoolley.co.uk, or by commenting at the bottom of this post.
Thank you for reading. 🙂