Common Proxying Issues

Everything on this page is taken from --> https://blog.nviso.eu/2020/11/19/proxying-android-app-traffic-common-issues-checklist/#check8

• Is your proxy configured on the device?

• Is Burp listening on all interfaces?

• Can your device connect to your proxy?

• Can you proxy HTTP traffic?

• Is your Burp certificate installed on the device?

• Is your Burp certificate installed as a root certificate?

• Does your Burp certificate have an appropriate lifetime?

• Is TLS Pass Through disabled?

• Is the application proxy aware?

• Is the application using custom ports?

• Is the application using SSL pinning?

  • Pinning through networkSecurityConfig

  • Pinning through OkHttp

  • Pinning through Obfuscated OkHttp in obfuscated apps

  • Pinning through various libraries

  • Pinning in third party app frameworks (Flutter, Xamarin, Unity)

Update: Sven Schleier also created a blogpost on this with some awesome visuals and graphs, so check that out as well!

Setting up the device

First, we need to make sure everything is set up correctly on the device. These steps apply regardless of the application you’re trying to MitM.

Is your proxy configured on the device?

An obvious first step is to configure a proxy on the device. The UI changes a bit depending on your Android version, but it shouldn’t be too hard to find.

Sanity check Go to Settings > Connections > Wi-Fi, select the Wi-Fi network that you’re on, click Advanced > Proxy > Manual and enter your Proxy details:

Proxy host name: 192.168.1.100 Proxy port: 8080

Is Burp listening on all interfaces?

By default, Burp only listens on the local interface (127.0.0.1) but since we want to connect from a different device, Burp needs to listen on the specific interface that has joined the Wi-Fi network. You can either listen on all interfaces, or listen on a specific interface if you know which one you want. As a sanity check, I usually go for ‘listen on all interfaces’. Note that Burp has an API which may allow other people on the same Wi-Fi network to query your proxy and retrieve information from it.

Sanity check Navigate to http://192.168.1.100:8080 on your host computer. The Burp welcome screen should come up.

Solution In Burp, go to Proxy > Options > Click your proxy in the Proxy Listeners window > check ‘All interfaces’ on the Bind to Address configuration

Can your device connect to your proxy?

Some networks have host/client isolation and won’t allow clients to talk to each other. In this case, your device won’t be able to connect to the proxy since the router doesn’t allow it.

Sanity Check Open a browser on the device and navigate to http://192.168.1.100:8080 . You should see Burp’s welcome screen. You should also be able to navigate to http://burp in case you’ve already configured the proxy in the previous check.

Solution There are a few options here:

• Set up a custom wireless network where host/client isolation is disabled

• Host your proxy on a device that is accessible, for example an AWS ec2 instance

• Perform an ARP spoofing attack to trick the mobile device into believing you are the router

• Use adb reverse to proxy your traffic over a USB cable:

  • Configure the proxy on your device to go to 127.0.0.1 on port 8080

  • Connect your device over USB and make sure that adb devices shows your device

  • Execute adb reverse tcp:8080 tcp:8080 which sends all traffic received on <device>:8080 to <host>:8080

  • At this point, you should be able to browse to http://127.0.0.1:8080 and see Burp’s welcome screen

Can you proxy HTTP traffic?

The steps for HTTP traffic are typically much easier than HTTPS traffic, so a quick sanity check here makes sure that your proxy is set up correctly and reachable by the device.

Sanity check Navigate to http://neverssl.com and make sure you see the request in Burp. Neverssl.com is a website that doesn’t use HSTS and will never send you to an HTTPS version, making it a perfect test for plaintext traffic.

Solution

• Go over the previous checks again, something may be wrong

• Burp’s Intercept is enabled and the request is waiting for your approval

Is your Burp certificate installed on the device?

In order to intercept HTTPS traffic, your proxy’s certificate needs to be installed on the device.

Sanity check Go to Settings > Security > Trusted credentials > User and make sure your certificate is listed. Alternatively, you can try intercepting HTTPS traffic from the device’s browser.

Solution This is documented in many places, but here’s a quick rundown:

• Navigate to http://burp in your browser

• Click the ‘CA Certificate’ in the top right; a download will start

• Use adb or a file manager to change the extension from der to crt

  • adb shell mv /sdcard/Download/cacert.der /sdcard/Download/cacert.crt

• Navigate to the file using your file manager and open the file to start the installation

Is your Burp certificate installed as a root certificate?

Applications on more recent versions of Android don’t trust user certificates by default. A more thorough writeup is available in another blogpost. Alternatively, you can repackage applications to add the relevant controls to the network_security_policy.xml file, but having your root CA in the system CA store will save you a headache on other steps (such as third-party frameworks) so it’s my preferred method.

Sanity check Go to Settings > Security > Trusted credentials > System and make sure your certificate is listed.

Solution In order to get your certificate listed as a root certificate, your device needs to be rooted with Magisk

• Install the client certificate as normal (see previous check)

• Install the MagiskTrustUser module

• Restart your device to enable the module

• Restart a second time to trigger the file copy

Alternatively, you can:

• Make sure the certificate is in the correct format and copy/paste it to the /system/etc/security/cacerts directory yourself. However, for this to work, your /system partition needs to be writable. Some rooting methods allow this, but it’s very dirty and Magisk is just so much nicer. It’s also a bit tedious to get the certificate in the correct format.

• Modify the networkSecurityConfig to include user certificates as trust anchors (see further down below). It’s much nicer to have your certificate as a system certificate though, so I rarely take this approach.

Does your Burp certificate have an appropriate lifetime?

Google (and thus Android) is aggressively shortening the maximum accepted lifetime of leaf certificates. If your leaf certificate’s expiration date is too far ahead in the future, Android/Chrome will not accept it. More information can be found in this blogpost.

Sanity check Connect to your proxy using a browser and investigate the certificate lifetime of both the root CA and the leaf certificate. If they’re shorter than 1 year, you’re good to go. If they’re longer, I like to play it safe and create a new CA. You can also use the latest version of the Chrome browser on Android to validate your certificate lifetime. If something’s wrong, Chrome will display the following error: ERR_CERT_VALIDITY_TOO_LONG

Solution There are two possible solutions here:

• Make sure you have the latest version of Burp installed, which reduces the lifetime of generated leaf certificates

• Make your own root CA that’s only valid for 365 days. Certificates generated by this root CA will also be shorter than 365 days. This is my preferred option, since the certificate can be shared with team members and be installed on all devices used during engagements.

Is TLS Pass Through disabled?

Burp allows you to configure certain domains which will not be MitM’d. This. is a setting called “TLS Passthrough” and you can either configure custom domains, or allow Burp to automatically add domains in case the client renegotiation failed.

Sanity check Go to Proxy > Options and scroll down to TLS Pass Through. Make sure that any domain you are trying to MITM is not listed, and also that the option to automatically add domains is not enabled.

Solution If the setting to automatically add entries is enabled, make sure you disable it.

If your domain is listed, make sure you remove it from the list or click the ‘Enabled’ flag to disable it. If your domain is listed, it was probably the result of a TLS negotiation failure in the past, and you most likely need to fix another issue, such as bypassing SSL pinning or correctly configuring your certificate. Now that you removed the domain from TLS Pass Through, go back to the start of the SSL checks of this guide and test again.

Setting up the application

Now that the device is ready to go, it’s time to take a look at application specifics.

Is the application proxy aware?

Many applications simply ignore the proxy settings of the system. Applications that use standard libraries will typically use the system proxy settings, but applications that rely on interpreted language (such as Xamarin and Unity) or are compiled natively (such as Flutter) usually require the developer to explicitly program proxy support into the application.

Sanity check When running the application, you should either see your HTTPS data in Burp’s Proxy tab, or you should see HTTPS connection errors in Burp’s Event log on the Dashboard panel. Since the entire device is proxied, you will see many blocked requests from applications that use SSL Pinning (e.g. Google Play), so see if you can find a domain that is related to the application. If you don’t see any relevant failed connections, your application is most likely proxy unaware.

As an additional sanity check, you can see if the application uses a third party framework. If the app is written in Flutter it will definitely be proxy unaware, while if it’s written in Xamarin or Unity, there’s a good chance it will ignore the system’s proxy settings.

• Decompile with apktool

  • apktool d myapp.apk

• Go through known locations

  • Flutter: myapp/lib/arm64-v8a/libflutter.so

  • Xamarin: myapp/unknown/assemblies/Mono.Android.dll

  • Unity: myapp/lib/arm64-v8a/libunity.so

Solution There are a few things to try:

• Use ProxyDroid (root only). Although it’s an old app, it still works really well. ProxyDroid uses iptables in order to forcefully redirect traffic to your proxy

• Set up a custom hotspot through a second wireless interface and use iptables to redirect traffic yourself. You can find the setup on the mitmproxy documentation, which is another useful HTTP proxy. The exact same setup works with Burp.

In both cases, you have moved from a ‘proxy aware’ to a ‘transparent proxy’ setup. There are two things you must do:

• Disable the proxy on your device. If you don’t do this, Burp will receive both proxied and transparent requests, which are not compatible with each other.

• Configure Burp to support transparent proxying via Proxy > Options > active proxy > edit > Request Handling > Support invisible proxying

Perform the sanity check again to now hopefully see SSL errors in Burp’s event log.

Is the application using custom ports?

This only really applies if your application is not proxy aware. In that case, you (or ProxyDroid) will be using iptables to intercept traffic, but these iptables rules only target specific ports. In the ProxyDroid source code, you can see that only ports 80 (HTTP) and 443 (HTTPS) are targeted. If the application uses a non-standard port (for example 8443 or 8080), it won’t be intercepted.

Sanity check This one is a bit more tricky. We need to find traffic that is leaving the application that isn’t going to ports 80 or 443. The best way to do this is to listen for all traffic leaving the application. We can do this using tcpdump on the device, or on the host machine in case you are working with a second Wi-Fi hotspot.

Run the following command on an adb shell with root privileges:

tcpdump -i wlan0 -n -s0 -v

You will see many different connections. Ideally, you should start the command, open the app and stop tcpdump as soon as you know the application has made some requests. After some time, you will see connections to a remote host with a non-default port. In the example below, there are multiple connections to 192.168.2.70 on port 8088:

Alternatively, you can send the output of tcpdump to a pcap by using tcpdump -i wlan0 -n -s0 -w /sdcard/output.pcap. After retrieving the output.pcap file from the device, it can be opened with WireShark and inspected:

Solution

If your application is indeed proxy unaware and communicating over custom ports, ProxyDroid won’t be able to help you. ProxyDroid doesn’t allow you to add custom ports, though it is an open-source project and a PR for this would be great. This means you’ll have to use iptables manually.

• Either you set up a second hotspot where your host machine acts as the router, and you can thus perform a MitM

• Or you use ARP spoofing to perform an active MitM between the router and the device

• Or you can use iptables yourself and forward all the traffic to Burp. Since Burp is listening on a separate host, the nicest solution is to use adb reverse to map a port on the device to your Burp instance. This way you don’t need to set up a separate hotspot, you just need to connect your device over USB.

  • On host: adb reverse tcp:8080 tcp:8080

  • On device, as root: iptables -t nat -A OUTPUT -p tcp -m tcp --dport 8088 -j REDIRECT --to-ports 8080

Is the application using SSL pinning?

At this point, you should be getting HTTPS connection failures in Burp’s Event log dashboard. The next step is to verify if SSL pinning is used, and disable it. Although many Frida scripts claim to be universal root bypasses, there isn’t a single one that even comes close. Android applications can be written in many different technologies, and only a few of those technologies are typically supported. Below you can find various ways in which SSL pinning may be implemented, and ways to get around it.

Note that some applications have multiple ways to pin a specific domain, and you may have to combine scripts in order to disable all of the SSL pinning.

Pinning through android:networkSecurityConfig

Android allows applications to perform SSL pinning by using the network_security_config.xml file. This file is referenced in the AndroidManifext.xml and is located in res/xml/. The name is usually network_security_config.xml but it doesn’t have to be. As an example application, the Microsoft Authenticator app has the following two pins defined:

Solution Use any of the normal universal bypass scripts:

• Run Objection and execute the android sslpinning disable command

• Use Frida codeshare: frida -U --codeshare akabe1/frida-multiple-unpinning -f be.nviso.app

• Remove the networkSecurityConfig setting in the AndroidManifest by using apktool d and apktool b. Usually much faster to do it through Frida and only rarely needed.

Pinning through OkHttp

Another popular way of pinning domains is through the OkHttp library. You can do a quick validation by grepping for OkHttp and/or sha256. You will most likely find references (or even hashes) relating to OkHttp and whatever is being pinned:

Solution Use any of the normal universal bypass scripts:

• Run Objection and execute the android sslpinning disable command

• Use Frida codeshare: frida -U --codeshare akabe1/frida-multiple-unpinning -f be.nviso.app

• Decompile the apk using apktool, and modify the pinned domains. By default, OkHttp will allow connections that are not specifically pinned. So if you can find and modify the domain name that is pinned, the pinning will be disabled. Using Frida is much faster though, so this approach is rarely taken.

Pinning through OkHttp in obfuscated apps

Universal pinning scripts may work on obfuscated applications since they hook on Android libraries which can’t be obfuscated. However, if an application is using something else than a default Android Library, the classes will be obfuscated and the scripts will fail to find the correct classes. A good example of this is OkHttp. When an application is using OkHttp and has been obfuscated, you’ll have to figure out the obfuscated name of the CertificatePinner.Builder class. You can see below that obfuscated OkHttp was used by searching on the same sha256 string. This time, you won’t see nice OkHttp class references, but you will typically still find string references and maybe some package names as well. This depends on the level of obfuscation of course.

Solution You’ll have to write your own Frida script to hook the obfuscated version of the CertificatePinner.Builder class. I have written down the steps to easily find the correct method, and create a custom Frida script in this blogpost.

Pinning through various libraries

Instead of using the networkSecurityConfig or OkHttp, developers can also perform SSL pinning using many different standard Java classes or imported libraries. Additionally, some Java based third party app such as the PhoneGap or AppCelerator frameworks provide specific functions to the developer to add pinning to the application.

There are many ways to do it programmatically, so your best bet is to just try various anti-pinning scripts and at least figure out what kind of methods are being triggered so that you have information on the app, after which you may be able to further reverse-engineer the app to figure out why interception isn’t working yet.

Solution Try as many SSL pinning scripts you can find, and monitor their output. If you can identify certain classes or frameworks that are used, this will help you in creating your own custom SSL pinning bypasses specific for the application.

• Run Objection and execute the android sslpinning disable command

• Use Frida scripts. Many of these have overlapping functionality, but you never know. Note how many of these claim to be universal

  • https://codeshare.frida.re/@akabe1/frida-multiple-unpinning/

  • https://codeshare.frida.re/@pcipolloni/universal-android-ssl-pinning-bypass-with-frida/

  • https://codeshare.frida.re/@sowdust/universal-android-ssl-pinning-bypass-2/

  • https://codeshare.frida.re/@masbog/frida-android-unpinning-ssl/

  • https://codeshare.frida.re/@segura2010/android-certificate-pinning-bypass/

  • https://codeshare.frida.re/@akabe1/frida-universal-pinning-bypasser/

Pinning in third party app frameworks

Third party app frameworks will have their own low-level implementation for TLS and HTTP and default pinning bypass scripts won’t work. If the app is written in Flutter, Xamarin or Unity, you’ll need to do some manual reverse engineering.

Figuring out if a third party app framework is used As mentioned in a previous step, the following files are giveaways for either Flutter, Xamarin or Unity:

• Flutter: myapp/lib/arm64-v8a/libflutter.so

• Xamarin: myapp/unknown/assemblies/Mono.Android.dll

• Unity: myapp/lib/arm64-v8a/libunity.so

Pinning in Flutter applications

Flutter is proxy-unaware and doesn’t use the system’s CA store. Every Flutter app contains a full copy of trusted CAs which is used to validate connections. So while it most likely isn’t performing SSL pinning, it still won’t trust the root CA’s on your device and thus interception will not be possible. More information is available in the blogposts mentioned below.

Solution Follow my blog post for either ARMv7 (x86) or ARMv64 (x64)

Pinning in Xamarin and Unity applications

Xamarin/Unity applications usually aren’t too difficult, but they do require manual reverse engineering and patching. Xamarin/Unity applications contain .dll files in the assemblies/ folder and these can be opened using .NET decompilers. My favorite tool is DNSpy which also allows you to modify the dll files.

Solution No blog post on this yet, sorry. The steps are as follows:

• Extract apk using apktool and locate .dll files

• Open .dll files using DNSpy and locate HTTP pinning logic

• Modify logic either by modifying the C# code or the IL

• Save the modified module

• Overwrite the .dll file with the modified version

• Repackage and resign the application

• Reinstall the application and run