<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Jonathan Cooper's Blog]]></title><description><![CDATA[I’m a cybersecurity consultant who codes. I help agile teams deliver secure digital experiences to their customers]]></description><link>https://joncooperworks.com</link><generator>RSS for Node</generator><lastBuildDate>Fri, 10 Apr 2026 17:30:17 GMT</lastBuildDate><atom:link href="https://joncooperworks.com/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Hacking HTTP with HTTPfuzz]]></title><description><![CDATA[So you’ve been given a web app to pentest. Maybe it’s a banking app or a document workflow system. Either way, you need to make sure it’s done safely. Modern web applications have a large attack surface, and testing everything by hand is inefficient....]]></description><link>https://joncooperworks.com/hacking-http-with-httpfuzz-1</link><guid isPermaLink="true">https://joncooperworks.com/hacking-http-with-httpfuzz-1</guid><category><![CDATA[Go Language]]></category><category><![CDATA[hacking]]></category><category><![CDATA[http]]></category><category><![CDATA[Testing]]></category><dc:creator><![CDATA[Jonathan Cooper]]></dc:creator><pubDate>Sat, 23 Jan 2021 14:06:49 GMT</pubDate><content:encoded><![CDATA[<p>So you’ve been given a web app to pentest. Maybe it’s a banking app or a document workflow system. Either way, you need to make sure it’s done safely. Modern web applications have a large attack surface, and testing everything by hand is inefficient. That’s where fuzzers come in handy. Fuzzers allow you to generate new inputs based on a seed and pass them to a program. Fuzzing can quickly show areas that are worth further examination.</p>
<p>I’m going to walk you through finding bugs in the <a target="_blank" href="http://www.dvwa.co.uk/">Damn Vulnerable Web App</a> (DVWA) with <a target="_blank" href="https://github.com/JonCooperWorks/httpfuzz">HTTPfuzz</a>, but you can apply this steps to any target as long as you have permission from the owner. <a target="_blank" href="https://github.com/JonCooperWorks/httpfuzz">HTTPfuzz</a> is a flexible HTTP fuzzer written in Go. It can fuzz any part of a request: multipart file uploads, multipart form fields, text request bodies, directories, filenames and URL query parameters.</p>
<p>Attacking services without consent is illegal in most countries. You can follow along using <a target="_blank" href="https://hub.docker.com/r/vulnerables/web-dvwa">Docker</a> on your computer without risking your freedom. Be sure to stop the container when you’re done using it, and only bind it to localhost to prevent yourself from getting hacked.</p>
<h2 id="caveat-emptor">Caveat Emptor</h2>
<p>A fuzzer is a very clumsy tool. It can knock systems offline, lock user accounts, fill up databases and generally annoy our friends on the blue team. I’d recommend only running a fuzzer against a development instance of the app. It’s useful for finding bugs before the software goes to production. Don’t run a fuzzer on a shared application unless you’re sure your inputs won’t cause damage and <strong>you have permission from the operations team</strong>. It’s best to set a delay between requests if you’re running HTTPfuzz against a shared application, and be sure to check how many requests you’ll send before firing your lasers.</p>
<h2 id="what-is-httpfuzz">What is HTTPFuzz?</h2>
<p>HTTPfuzz is a flexible CLI HTTP fuzzer written in Go. It supports fuzzing any part of a request body, including multipart file uploads, JSON fields, HTTP Headers and URL parameters. HTTPfuzz can be extended with Go <a target="_blank" href="https://golang.org/pkg/plugin/">plugins</a> for better integration with your pentesting workflow.</p>
<h3 id="plugins">Plugins</h3>
<p>HTTPfuzz plugins are simply Go plugins that export a method named New that returns an implementation of HTTPfuzz’s Listener interface. Everything you need to know to write plugins is in the gist below.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="ad0c8fd4e22303cb8a864dc4724f58d1"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/ad0c8fd4e22303cb8a864dc4724f58d1" class="embed-card">https://gist.github.com/JonCooperWorks/ad0c8fd4e22303cb8a864dc4724f58d1</a></div><p>A Listener receives a stream of Results from the fuzzer. A Result contains the HTTP response from the target, the corresponding request, the payload, payload location and the time the request took. Plugins can use this to save requests and responses to a database, check for vulnerabilities and anything else you can think of.</p>
<h2 id="file-uploads">File Uploads</h2>
<p>Many apps allow users to upload files for all kinds of reasons: photo galleries, documents, scanning cheques and much more. File uploads expose tons of attack surface. XSS, path injection, introducing malware into a network, remote code execution and so much more is available to you by bugs in file uploads. DVWA is no exception.</p>
<p><img src="https://cdn-images-1.medium.com/max/2760/1*iZJ_DyLxWyWw5iMFQOhRYw.png" alt="File upload in DVWA. Do you notice anything interesting about the filename?" /><em>File upload in DVWA. Do you notice anything interesting about the filename?</em></p>
<h3 id="automatically-generated-files">Automatically Generated Files</h3>
<p>The easiest way to check which file types are allowed is simply uploading different kinds of files and looking at the responses. If a website says it only accepts photos, try uploading a PHP file or an executable and see what happens. For example, JPEG photos always start with the bytes FF D8 FF. You can use a hex editor like <a target="_blank" href="https://ridiculousfish.com/hexfiend/">Hex Fiend</a> to see the byte patterns at the beginning of every file of the same time.</p>
<p>If the whitelisting is done correctly, it will check the first few bytes of the uploaded file against a list of expected <a target="_blank" href="https://www.garykessler.net/library/file_sigs.html">file signatures</a>. HTTPfuzz generates files by putting header bytes for the file type at the top of a byte array filled with random bytes. These generated files will probably not be valid, which is why you’d use user-supplied payloads once you’ve determined how restrictive the validation is.</p>
<p>We can combine this with a simple plugin that tells us when a file has been successfully uploaded to let us enumerate the allowed file types.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="46e610fe4c815406a4cd8e6542030193"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/46e610fe4c815406a4cd8e6542030193" class="embed-card">https://gist.github.com/JonCooperWorks/46e610fe4c815406a4cd8e6542030193</a></div><p>Build the plugin and send off the automatically generated file payloads. We’ll be able to see what’s allowed based on what is successfully uploaded.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="ee875bdc29e193fee6d5ab73a4e6f55e"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/ee875bdc29e193fee6d5ab73a4e6f55e" class="embed-card">https://gist.github.com/JonCooperWorks/ee875bdc29e193fee6d5ab73a4e6f55e</a></div><p>Running this reveals that DVWA does not perform any filtering at all on uploaded files: it’s a free for all.</p>
<h3 id="user-supplied-payloads">User-Supplied Payloads</h3>
<p>Now that we know what kind of files are allowed, it’s time to see if any vulnerabilities can be exploited. We should see if we can exploit bugs in how DVWA handles filenames and types based on what we discovered with the automatically generated files. Since we know DVWA doesn’t perform any filtering, we should try for PHP code execution.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="87b1f52fdbb4c376ea33a8ea4de5ef44"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/87b1f52fdbb4c376ea33a8ea4de5ef44" class="embed-card">https://gist.github.com/JonCooperWorks/87b1f52fdbb4c376ea33a8ea4de5ef44</a></div><div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="87b1f52fdbb4c376ea33a8ea4de5ef44"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/87b1f52fdbb4c376ea33a8ea4de5ef44" class="embed-card">https://gist.github.com/JonCooperWorks/87b1f52fdbb4c376ea33a8ea4de5ef44</a></div><div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="87b1f52fdbb4c376ea33a8ea4de5ef44"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/87b1f52fdbb4c376ea33a8ea4de5ef44" class="embed-card">https://gist.github.com/JonCooperWorks/87b1f52fdbb4c376ea33a8ea4de5ef44</a></div><h3 id="php-code-execution">PHP Code Execution</h3>
<p>One of the payloads was a PHP program that would display information about the server’s PHP environment. Since DVWA has a file inclusion vulnerability, we can execute our payload by passing the saved payload’s filename to the vulnerable “page” URL parameter.</p>
<p><img src="https://cdn-images-1.medium.com/max/4292/1*QWS2gO_2Y8IRGUZa8YhH6w.png" alt="Using the local file inclusion vulnerability to trigger our payload." /><em>Using the local file inclusion vulnerability to trigger our payload.</em></p>
<p>We could stop here since we owned the box, but I want to show you some more HTTPfuzz features.</p>
<h2 id="command-injection">Command Injection</h2>
<p>You can mark targets in text request bodies with the delimiter character. By default, it’s “`”. The command injection challenge sends an IP address via POST request to a vulnerable ping function. You can often find vulnerabilities like this on routers and online vulnerability scanners. Consider the following request:</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="4d8c12b936ea494294c03e63a611569e"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/4d8c12b936ea494294c03e63a611569e" class="embed-card">https://gist.github.com/JonCooperWorks/4d8c12b936ea494294c03e63a611569e</a></div><p>We’re trying to execute code on the DVWA. We’ll try the Unix and Windows command injection payloads from the <a target="_blank" href="https://github.com/payloadbox/command-injection-payload-list">command-injection-payload-list</a> and examine the results in an intercepting proxy to see if any of these give us a clue how to exploit the vulnerability.</p>
<h2 id="brute-force">Brute Force</h2>
<p>HTTPfuzz makes it easy to brute force your way into valid accounts. The DVWA has a brute force challenge that accepts the username and password as GET parameters. Consider the HTTP request below.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="8702dd4fe2382ac6181a037af2e6f161"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/8702dd4fe2382ac6181a037af2e6f161" class="embed-card">https://gist.github.com/JonCooperWorks/8702dd4fe2382ac6181a037af2e6f161</a></div><p>First, we’ll need a wordlist with common passwords. I’ll use <a target="_blank" href="https://github.com/brannondorsey/naive-hashcat/releases/download/data/rockyou.txt">rockyou.txt</a> for this demonstration. We’ll also need to differentiate between successful and unsuccessful login responses. An easy way to do this is to submit an invalid password and observe the response.</p>
<p><img src="https://cdn-images-1.medium.com/max/2752/1*B909PN9LzRShkNGXjIep4A.png" alt="The message “Username and/or password incorrect.” always appears when login fails." /><em>The message “Username and/or password incorrect.” always appears when login fails.</em></p>
<p>Trying to send a bad password gives an error message. That’s good enough for us. We can create a HTTPfuzz plugin that returns the payload used when that error message isn’t in the page. That payload should be the password.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="d43c739020a5d98f51ce831edd5c8f6e"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/d43c739020a5d98f51ce831edd5c8f6e" class="embed-card">https://gist.github.com/JonCooperWorks/d43c739020a5d98f51ce831edd5c8f6e</a></div><p>Build the plugin and pass it to HTTPfuzz to see if admin’s password is in rockyou.txt. Since the password is in rockyou.txt, the plugin will print it so we can solve this challenge.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="916fb83e8420c4ad13540f077ae984c4"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/916fb83e8420c4ad13540f077ae984c4" class="embed-card">https://gist.github.com/JonCooperWorks/916fb83e8420c4ad13540f077ae984c4</a></div><p>Running this attack shows that the admin’s password is indeed in rockyou.txt. This attack may seem contrived, but many appliances and programs have known default passwords. The <a target="_blank" href="https://en.wikipedia.org/wiki/Mirai_(malware">Mirai botnet</a>) spread by brute forcing known default router passwords via telnet.</p>
<h2 id="get-httpfuzz">Get httpfuzz</h2>
<p>You can check out <a target="_blank" href="https://github.com/JonCooperWorks/httpfuzz">httpfuzz</a> on GitHub. It’s written in Go and GPLv3 licensed. It runs on Windows, Linux and macOS. It’s a versatile HTTP testing tool that can perform many attacks, like dirbuster style directory brute force attacks and HTTP header fuzzing.</p>
]]></content:encoded></item><item><title><![CDATA[Judas: Phishing Resurrected]]></title><description><![CDATA[If you’ve been reading my blog since I started writing on Medium, you’ll remember Judas, the pluggable open-source phishing proxy. I wrote Judas to prove a point on an engagement once, and unfortunately neglected it afterwards. (Side note: Go’s compr...]]></description><link>https://joncooperworks.com/judas-phishing-resurrected</link><guid isPermaLink="true">https://joncooperworks.com/judas-phishing-resurrected</guid><dc:creator><![CDATA[Jonathan Cooper]]></dc:creator><pubDate>Mon, 05 Oct 2020 14:52:06 GMT</pubDate><content:encoded><![CDATA[<p>If you’ve been reading my blog since I started writing on <a target="_blank" href="https://medium.com/@joncooperworks">Medium</a>, you’ll remember <a target="_blank" href="https://github.com/JonCooperWorks/judas">Judas</a>, the pluggable open-source phishing proxy. I wrote Judas to prove a point on an engagement once, and unfortunately neglected it afterwards. (Side note: Go’s comprehensive standard library makes it easy to toss together a proof of concept on an engagement). I’ve had a lot more time to write code in <a target="_blank" href="https://jis.gov.jm/islandwide-curfew-extended-to-october-7/">lockdown</a> so I decided to show Judas some love.</p>
<h1 id="what-is-judas">What is Judas?</h1>
<p>Judas is a pluggable reverse proxy for red team phishing engagements based on Go’s <a target="_blank" href="https://golang.org/pkg/net/http/httputil/#ReverseProxy">httputil.ReverseProxy</a>. Simply point it at the target website, give it a domain for the SSL certificate and you’re on your way. Judas makes a byte-for-byte copy of the original requests and responses making it impossible to tell the difference between a phishing site and the original without checking the URL bar. Victims will be able to log into the target and use it like they normally do.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1601897486198/5yL7CXhlg.png" alt="Judas.png" /></p>
<blockquote>
<p>Judas cloning <a target="_blank" href="https://www.targetpractice.network/">targetpractice.network</a> on localhost:9000. </p>
<p>(I own <a target="_blank" href="https://www.targetpractice.network/">targetpractice.network</a>, don’t use Judas on websites you don’t have permission to attack).</p>
</blockquote>
<p>This is made possible by supplying functions for the <a target="_blank" href="https://golang.org/pkg/net/http/httputil/#ReverseProxy">ReverseProxy</a> struct that rewrite requests and responses on the fly. <a target="_blank" href="https://golang.org/pkg/net/http/httputil/#ReverseProxy">ReverseProxy</a> is versatile: you can use it to build Web Application Firewalls, patch vulnerabilities in services you can’t update and much more. It’s worth adding to your toolkit no matter which team you’re on.</p>
<h1 id="stealth-goodies">Stealth Goodies</h1>
<p>Judas comes with some features to help us hide from the target server and victims. We want to stay hidden from the blue team as long as possible to increase our chances of compromising a high value account.</p>
<h2 id="automatic-lets-encrypt">Automatic Let’s Encrypt</h2>
<p>We’ve spent the last few years conditioning people to trust that padlock in their URL bar. Thanks to <a target="_blank" href="https://letsencrypt.org/">Let’s Encrypt</a>, we can get free SSL certificates automatically. Simply tell Judas your phishing site’s domain and it’ll do the rest.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="f793340b79fe0a6b999c6c9edc8b951c"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/f793340b79fe0a6b999c6c9edc8b951c" class="embed-card">https://gist.github.com/JonCooperWorks/f793340b79fe0a6b999c6c9edc8b951c</a></div><h2 id="proxy-support">Proxy Support</h2>
<p>Judas is able to use a HTTP or SOCKS5 proxy. You can hide requests from your target using a proxy, log them in case a client wants to see what exactly you took, get an IP address close to the target and much more. You can do multiple of these at once with <a target="_blank" href="https://github.com/haad/proxychains">proxychains</a>.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="7ea691bd71f3cdef20bd035f0c21e6e2"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/7ea691bd71f3cdef20bd035f0c21e6e2" class="embed-card">https://gist.github.com/JonCooperWorks/7ea691bd71f3cdef20bd035f0c21e6e2</a></div><h2 id="http-request-header-rewriting">HTTP Request Header Rewriting</h2>
<p>Judas automatically rewrites HTTP request headers before sending them to the target website. It replaces the our phishing site in the <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer">Referer</a> and <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Origin">Origin</a> headers with an appropriate value and removes the user agent if the user didn’t send one to prevent Go’s default <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent">User Agent</a> from showing up in logs and giving us away.</p>
<h2 id="http-response-header-rewriting">HTTP Response Header Rewriting</h2>
<p>Judas automatically rewrites HTTP responses before they come back to the client. It removes any <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy">Content-Security Policy</a> and <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection">anti-XSS</a> headers on the response to prevent them from catching the phishing proxy and sending it to a <a target="_blank" href="https://report-uri.com/">report-URI</a> (sorry blue team). It also rewrites the <a target="_blank" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Location">Location</a> header to stop redirects with the full URL from sending the victim back to the original site.</p>
<h1 id="plugins">Plugins</h1>
<p>Real life is complicated. Maybe you need to phish a lot of people, or maybe you want to do something on behalf of a phished user because they have 2-Factor authentication enabled. Judas lets you use Go <a target="_blank" href="https://golang.org/pkg/plugin/">plugins</a> to:</p>
<ul>
<li>Modify requests after they’ve been sent by the victim to enable attacks like replacing an admin’s request with a request to create a user for us</li>
<li>Modify responses before they’re received by the victim so we can hide the results of attacks or hit victims with an exploit kit.</li>
<li>Process request — response HTTP exchanges from victims in real time to programmatically extract sensitive information or store them in <a target="_blank" href="https://cosmos.azure.com/">Azure CosmosDB</a> to look at later.</li>
</ul>
<h2 id="requesttransformers">RequestTransformers</h2>
<p>A RequestTransformer is a function in a Go plugin named “RequestTransformer” that modifies an http.Request from a victim in place before it’s sent to the target. It should be fast so users don’t get suspicious or turned off by delays. You can use them to hijack legitimate requests. For example, a RequestTransformer could replace an admin users’s request with one that creates a user for us to gain access to protected systems.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="5dc47011bf4517e71741741f38b3bd6e"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/5dc47011bf4517e71741741f38b3bd6e" class="embed-card">https://gist.github.com/JonCooperWorks/5dc47011bf4517e71741741f38b3bd6e</a></div><h2 id="responsetransformers">ResponseTransformers</h2>
<p>A ResponseTransformer is a function in a Go plugin named “ResponseTransformer” that modifies an http.Response from a target in place before it’s received by the victim. You can use them to hide what you’ve done from a target, or hit victims with an exploit kit.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="5b299b6ef990b878f69b99b5d289ac88"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/5b299b6ef990b878f69b99b5d289ac88" class="embed-card">https://gist.github.com/JonCooperWorks/5b299b6ef990b878f69b99b5d289ac88</a></div><h2 id="listeners">Listeners</h2>
<p>A listener runs in its own goroutine and receives request — response HTTP exchanges from victims in real time to allow us to extract sensitive information. The plugin loader looks for a function called “New” that returns a Judas Listener in a Go plugin.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="c8dd47dcaa4957a19e2e0ac6e5ba600e"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/c8dd47dcaa4957a19e2e0ac6e5ba600e" class="embed-card">https://gist.github.com/JonCooperWorks/c8dd47dcaa4957a19e2e0ac6e5ba600e</a></div><h2 id="using-plugins">Using Plugins</h2>
<p>Once you’ve made a plugin, <a target="_blank" href="https://golang.org/pkg/plugin/">compile it</a> and load it using command line arguments. Multiple plugins can be loaded be separating the plugin file paths with colons (“:”). There’s an <a target="_blank" href="https://github.com/JonCooperWorks/judas/blob/master/examples/searchloggingplugin/searchloggingplugin.go">example plugin</a> with all 3 operations implemented on Judas's Github page.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="609022daf4bffeec8eecea327453b673"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/609022daf4bffeec8eecea327453b673" class="embed-card">https://gist.github.com/JonCooperWorks/609022daf4bffeec8eecea327453b673</a></div><h1 id="get-judas-on-github">Get Judas on Github</h1>
<p>Judas’s source code is on <a target="_blank" href="https://github.com/JonCooperWorks/judas">Github</a>. It’s MIT licensed, so feel free to modify it, fork it or whatever you see fit to do with it. Just don’t use it for crime. I’m not responsible for anything illegal you do with Judas, and I will laugh at you when cybercrime authorities in your country break your door down. Use your skills to <a target="_blank" href="http://hackerone.com/">make the world a better place</a> instead of spreading misery.</p>
<div class="embed-wrapper"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a class="embed-card" href="https://github.com/JonCooperWorks/judas">https://github.com/JonCooperWorks/judas</a></div>
]]></content:encoded></item><item><title><![CDATA[Printing Money With TD Ameritrade's API]]></title><description><![CDATA[We’ve all heard about algorithmic trading in the news. It sounds great! You think of a program making money for you while you sip Mai Tais and smoke the finest ganja on the beach in Jamaica. I’m going to show you how you can do it yourself with TD Am...]]></description><link>https://joncooperworks.com/printing-money-with-td-ameritrades-api-1</link><guid isPermaLink="true">https://joncooperworks.com/printing-money-with-td-ameritrades-api-1</guid><category><![CDATA[golang]]></category><dc:creator><![CDATA[Jonathan Cooper]]></dc:creator><pubDate>Thu, 13 Aug 2020 05:00:00 GMT</pubDate><content:encoded><![CDATA[<p>We’ve all heard about algorithmic trading in the <a target="_blank" href="https://www.ft.com/content/fdc1c064-1142-11e9-a581-4ff78404524e">news</a>. It sounds great! You think of a program making money for you while you sip Mai Tais and smoke the finest ganja on the beach in Jamaica. I’m going to show you how you can do it yourself with TD Ameritrade and a laptop.</p>
<div class="embed-wrapper"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a class="embed-card" href="https://github.com/JonCooperWorks/go-tdameritrade">https://github.com/JonCooperWorks/go-tdameritrade</a></div>
<p>Before we start, you need to know that great reward comes with great risk. Don’t do this with money you’re not willing to lose.</p>
<img alt="Image for post" src="https://miro.medium.com/max/8064/1*jvmUJODtOizO2KrOhoZ5WQ.jpeg" />

<blockquote>
<p>Sunrise in St. Elizabeth, <a target="_blank" href="https://www.instagram.com/jamaica/">Jamaica</a>. One of the best places in the world to spend your trading profits.</p>
</blockquote>
<h1 id="getting-api-keys">Getting API Keys</h1>
<p>The first thing you’ll need is a <a target="_blank" href="http://tdameritrade.com/">TD Ameritrade</a> account. You don’t need to be a US citizen to get an account, and the process is pretty easy. If you’re in Jamaica, you’ll have to send TD Ameritrade signed copies of the documents they give you via Fedex or DHL. After you’ve signed up for TD Ameritrade, head over to their <a target="_blank" href="https://developer.tdameritrade.com/">developer website</a> and sign up for a developer account. Create an app and specify it. TD’s developer website will not accept a <a target="_blank" href="https://www.oauth.com/oauth2-servers/redirect-uris/">Callback URL</a> that doesn’t begin with https, even if it’s localhost. You can test applications locally by removing the “https” with “http” in the URL bar after the redirect, or serving localhost over TLS.</p>
<img alt="Image for post" src="https://miro.medium.com/max/4964/1*TL9V9GRQV9izpvUFaEvTYA.png" />

<blockquote>
<p>The app Callback URL must begin with https://</p>
</blockquote>
<p>Once you’ve created your app, its <a target="_blank" href="http://oauth.com/oauth2-servers/client-registration/client-id-secret/">client ID</a> will be available to you in the dashboard. It seems you can only create one app per developer account.</p>
<img alt="Image for post" src="https://miro.medium.com/max/4764/1*XyTSm7XOQqXmXlRE8ramzA.png" />

<blockquote>
<p>The OAuth2 <a target="_blank" href="http://oauth.com/oauth2-servers/client-registration/client-id-secret/">Client ID</a> is available as the Consumer Key in the dashboard.</p>
</blockquote>
<p>You can use this client ID in any <a target="_blank" href="https://www.oauth.com/">OAuth2</a> client library to interact with TD Ameritrade’s API. If you use <a target="_blank" href="https://golang.org">Go</a>, I’ve written a <a target="_blank" href="https://github.com/JonCooperWorks/go-tdameritrade">TD Ameritrade client library</a> that handles authentication and interaction with the TD Ameritrade REST API, and if you use Python, Alex Golec wrote a <a target="_blank" href="https://github.com/alexgolec/tda-api">python wrapper</a>.</p>
<h1 id="api-documentation">API Documentation</h1>
<p>TD Ameritrade’s documentation is available on its <a target="_blank" href="https://developer.tdameritrade.com/">developer website</a>, but it is not as comprehensive as you may be used to from companies like Apple and Google. There are several surprises in their API that I’m going to tell you about to make your life easier.</p>
<h2 id="client-id-surprise">Client ID Surprise</h2>
<p>You have to append <code>@AMER.OAUTHAP</code> to your OAuth2 client ID, or it will return an error. This is only documented on their <a target="_blank" href="https://developer.tdameritrade.com/content/authentication-faq">Authentication FAQ</a> and <a target="_blank" href="https://developer.tdameritrade.com/content/getting-started">Getting Started Guide</a> in the example URLs.</p>
<img alt="Image for post" src="https://miro.medium.com/max/3052/1*LQDU4Osfy1CxInsx_e_HNQ.png" />

<blockquote>
<p>The dreaded “This may be due to a technical error, or the client application may be an attempt to fraudulently access your account.” error</p>
</blockquote>
<h2 id="papermoney-doesnt-work">PaperMoney Doesn’t Work</h2>
<p>TD Ameritrade has a world class demo trading platform, <a target="_blank" href="https://papermoney.thinkorswim.com/platform/index.html">PaperMoney</a>. It has real time data when you have a funded account with TD Ameritrade. It’s the best demo trading platform I’ve ever used. It’s also not available via the API. It’s not clear when, if ever, TD Ameritrade will make demo accounts available via the API. It’s best to test your strategy with the data using a backtesting library, but make sure that your orders would actually fill given the state of the liquidity in the order book at the time before going live.</p>
<h2 id="real-time-updates">Real Time Updates</h2>
<p>TD Ameritrade provides free <a target="_blank" href="https://developer.tdameritrade.com/content/streaming-data">real time market data</a> over a websockets API. Alex Golec’s <a target="_blank" href="https://github.com/alexgolec/tda-api">python wrapper</a> allows you to process the market data in real time using Python’s async feature.</p>
<h1 id="risk-management">Risk Management</h1>
<p>The most important skill when trading is risk management. This is even more vital when running an automated trading bot, since <a target="_blank" href="https://www.henricodolfing.com/2019/06/project-failure-case-study-knight-capital.html">computers can lose lots of money very quickly</a>. You’ll need to defend against hackers and trading losses.</p>
<img alt="Image for post" src="https://miro.medium.com/max/1058/1*Bphe18r986Kn7MlFoV6gZg.png" />

<blockquote>
<p>The bigger a loss, the harder it is to recover. Image from <a target="_blank" href="http://www.fusioninvesting.com/2011/02/investing-myths-gain-required-to-make-you-whole/">fusioninvesting.com</a></p>
</blockquote>
<h2 id="avoiding-margin-calls">Avoiding Margin Calls</h2>
<ul>
<li>Don’t run your trading bot on your main investment TD account. Create a new account and fund that with a smaller amount of money to trade on.</li>
<li>Ensure your bot has trade size and daily loss limits specified in dollars (<strong>not percentages</strong>) to prevent bugs from causing you to blow up your account. You can make these configurable so they can grow with your account.</li>
<li>If you’re trading options, futures or other leveraged securities, be sure to take the <strong>total risk</strong> into account when designing your limits. A leveraged position can be <a target="_blank" href="https://earlyretirementnow.com/2018/12/18/the-optionsellers-debacle/">very volatile</a>.</li>
<li>Monitor your account balance using TD Ameritrade’s apps. This will protect you from bugs in your code telling you you’re making millions when you’re really on your way to a <a target="_blank" href="https://tlc.thinkorswim.com/center/howToTos/thinkManual/Miscellaneous/Margin/Margin-Call">margin call</a>.</li>
<li>Run the bot in demo mode by default and create a switch to place live trades to stop yourself from accidentally running with real money when you think it’s paper money. For maximum assurance, have the bot require an explicit command to begin placing trades.</li>
<li>Ensure there is an easy external killswitch that will stop all trades immediately in case of runaway losses. My bot runs on an <a target="_blank" href="https://azure.microsoft.com/en-us/">Azure</a> VPS, and can be killed quickly by simply shutting the VPS off from the Azure Portal. This lets me kill the bot even if my VPN is down, or the program isn’t responding to kill signals from the OS.</li>
</ul>
<h2 id="hosting-and-security">Hosting and Security</h2>
<p>Your trading bot will have access to your money. Although you cannot withdraw money via the API, hackers can steal from you in other ways, like having you sell all your holdings and purchase some low liquidity, worthless securities that they hold at a higher price, like the <a target="_blank" href="https://hackernoon.com/alleged-hack-of-binance-linked-to-viacoin-pump-bb9066bf96bf">VIAcoin pump</a> on Binance.</p>
<ul>
<li>Do not give your bot server a public IP address. Use a VPN or a gateway like <a target="_blank" href="https://www.cloudflare.com/teams-access/">Cloudflare Access</a> to access your server. If you’re self-hosting a VPN, ensure the VPN and VPN server OS are always updated to the latest versions.</li>
<li>Use firewall rules to limit access to only the ports used by your bot, and only allow it to send traffic to TD Ameritrade and your OS’s update servers. If possible, limit traffic at the network level, like Azure’s <a target="_blank" href="https://docs.microsoft.com/en-us/azure/virtual-network/security-overview">Network Security Groups</a> or AWS’s <a target="_blank" href="https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html">Security Groups</a> instead of relying on a host-based firewall. Do not use unencrypted network protocols to control the bot. I’d recommend using HTTPS with TLSv1.2 or TLSv1.3.</li>
<li>Only install the bot and software necessary for it to run. I like Go because it can produce a single static binary that runs on a minimal Linux distro or Windows Server install.</li>
<li>Keep the <a target="_blank" href="https://owasp.org/www-project-top-ten/">OWASP Top 10</a> in mind when building the bot. Use modern software development frameworks that help mitigate against bugs like <a target="_blank" href="https://owasp.org/www-community/attacks/xss/">XSS</a> and <a target="_blank" href="https://owasp.org/www-community/attacks/csrf">CSRF</a>. Be sure to write automated tests for your code, especially your <strong>trading strategies and limits</strong> and ensure code can only be merged into your master branch after all its tests pass. Be sure to leverage static analysis to catch bugs where possible, and I’d recommend using a statically typed programming language so the compiler and type system can help you write accurate code.</li>
<li>Ensure you update your library dependencies regularly. Vulnerabilities in dependencies can compromise your bot and cost you money.</li>
<li>Ensure your bot server is monitored. Track your logs and system performance using an external log monitoring system, and set alarms for strange events, like new processes, or attempts to send network traffic to non-approved sources.</li>
<li>Use a cloud provider like <a target="_blank" href="https://azure.microsoft.com/en-us/">Microsoft Azure</a> or <a target="_blank" href="https://aws.amazon.com/">Amazon AWS</a>. They can likely provide better uptime and security than you can on hardware you operate. Your mileage may vary. Start with the smallest VPS and scale up as needed.</li>
<li>Use a dedicated device to access your trading bot. Install as little third party software as possible. Ideally, you’d only install a browser and the VPN client. Use TD Ameritrade’s website to monitor your performance.</li>
</ul>
<h1 id="my-setup">My Setup</h1>
<p>I run my bot on an Ubuntu 20.04 VPS on <a target="_blank" href="https://azure.microsoft.com/en-us/">Microsoft Azure</a> in a private subnet only accessible on port 443 via VPN using <a target="_blank" href="https://docs.microsoft.com/en-us/azure/virtual-network/security-overview">Azure Network Security Groups</a>. The bot web UI is only available via HTTPS, and the server is only able to send traffic to TD Ameritrade’s API server and the OS update servers.</p>
<p>The server has a <a target="_blank" href="https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview">managed identity</a> that it uses to access other services within the Azure account. Secrets like TLS certificates are stored in <a target="_blank" href="https://azure.microsoft.com/en-us/services/key-vault/">Azure Key Vault</a>. I monitor my server logs and performance with Azure <a target="_blank" href="https://docs.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview">Application Insights</a>, and I monitor traffic to and from the server’s subnet with Azure <a target="_blank" href="https://docs.microsoft.com/en-us/azure/network-watcher/traffic-analytics">Traffic Analytics</a>.</p>
<p>The bot is a single binary built in Go. It exposes a web UI on port 443 over HTTPS only (I don’t need automatic HTTP -&gt; HTTPS redirects) and does not use many non-stdlib libraries to reduce attack surface and dependencies to keep current. It is the only non-default program running on the VPS.</p>
<h1 id="trading-strategy">Trading Strategy</h1>
<p>This is the hard part. Your bot is useless unless it’s executing trades based on a profitable strategy. I’m not going to tell you how to come up with this part. Read about technical analysis and fundamental analysis. You can use external data, like news feeds and social media to influence your trading strategy. It’s best to automate a profitable trading strategy that already works for you manually. Read posts on <a target="_blank" href="http://reddit.com/r/algotrading/">/r/algotrading</a> for inspiration.</p>
<h1 id="trade-on-td-ameritrade-with-go">Trade on TD Ameritrade with Go</h1>
<p>If you made it this far, you’re probably really interested in algorithmic trading. I created <a target="_blank" href="https://github.com/JonCooperWorks/go-tdameritrade">go-tdameritrade</a>, a fork of Zachary Rice’s <a target="_blank" href="https://github.com/zricethezav/go-tdameritrade">go-tdameritrade</a> to make it easier to write trading bots. It’s free and open source. Use this library at your own risk. Automated trading can cause you to <a target="_blank" href="https://www.henricodolfing.com/2019/06/project-failure-case-study-knight-capital.html">lose money very quickly</a>.</p>
]]></content:encoded></item><item><title><![CDATA[Easy private networks with WireguardHTTPS]]></title><description><![CDATA[I’ve been experimenting with Wireguard as a VPN to protect my internet traffic from local snoopers and communicate between all my devices as if they were on the same network.



SSHing into an Ubuntu 20.04 Thinkpad on my home network via the VPN.

Wi...]]></description><link>https://joncooperworks.com/easy-private-networks-with-wireguardhttps</link><guid isPermaLink="true">https://joncooperworks.com/easy-private-networks-with-wireguardhttps</guid><category><![CDATA[vpn]]></category><category><![CDATA[Azure]]></category><category><![CDATA[golang]]></category><category><![CDATA[GitHub]]></category><dc:creator><![CDATA[Jonathan Cooper]]></dc:creator><pubDate>Thu, 23 Jul 2020 05:00:00 GMT</pubDate><content:encoded><![CDATA[<p>I’ve been experimenting with <a target="_blank" href="https://wireguard.com/">Wireguard</a> as a VPN to protect my internet traffic from local snoopers and communicate between all my devices as if they were on the same network.</p>
<img alt="Image for post" src="https://miro.medium.com/max/2268/1*J11YRfn6PSwdtE6w4NUO1w.png" />

<blockquote>
<p>SSHing into an Ubuntu 20.04 Thinkpad on my home network via the VPN.</p>
</blockquote>
<p>Wireguard was designed with mobile devices in mind. It uses battery-friendly cryptography and the protocol can handle endpoints that change IP address seamlessly. It is ideal for exposing local development servers on my laptop to my iPhone, but deploying configuration to a device is a manual and time-consuming process.</p>
<div class="embed-wrapper"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a class="embed-card" href="https://github.com/JonCooperWorks/wireguardhttps">https://github.com/JonCooperWorks/wireguardhttps</a></div>
<p><a target="_blank" href="https://github.com/JonCooperWorks/wireguardhttps">WireguardHTTPS</a> is a Wireguard access server written in <a target="_blank" href="https://golang.org/">Go</a> that allows users to log in with <a target="_blank" href="https://azure.microsoft.com/en-us/services/active-directory/">Azure AD</a> and manage access to the Wireguard VPN.</p>
<img alt="Image for post" src="https://miro.medium.com/max/1996/1*tM7dic8BGJSOVj5WqQqo1w.png" />

<blockquote>
<p>WireguardHTTPS generates configuration and creates QR codes for easy deployment to mobile devices.</p>
</blockquote>
<p>WireguardHTTPS is intentionally very simple. It only allows users to add devices to the VPN, regenerate device credentials and remove devices from the VPN. WireguardHTTPS does not store private keys: once a device has been created, future attempts to download its credentials will revoke the device’s access to the VPN and generate new credentials.</p>
<img alt="Image for post" src="https://miro.medium.com/max/1224/1*kum2oK8aoaC4lC25H_xZig.png" />

<blockquote>
<p>WireguardHTTPS only allows regenerating credentials and deleting a device.</p>
</blockquote>
<p>Devices are automatically assigned IP addresses in the subnet specified when setting up WireguardHTTPS. When a device is deleted, its IP address is released for reuse by a future device.</p>
<h1 id="architecture">Architecture</h1>
<p>WireguardHTTPS is made of 3 components to limit privileges required by the internet-facing HTTPS API while allowing it to control the higher privileged Wireguard interface and protect users from common web attacks like <a target="_blank" href="https://owasp.org/www-community/attacks/xss/">XSS</a>, <a target="_blank" href="https://owasp.org/www-community/attacks/Clickjacking">Clickjacking</a>, and <a target="_blank" href="https://owasp.org/www-community/attacks/csrf">CSRF</a>.</p>
<p>A root gRPC daemon, <a target="_blank" href="https://github.com/JonCooperWorks/wgrpcd">wgrpcd</a>, controls the Wireguard interface. Wgrpcd manages devices directly on the Wireguard interface using <a target="_blank" href="https://github.com/WireGuard/wgctrl-go">wgctrl</a> and responds to gRPC messages from other processes. It is not aware of IP address allocation, DNS or Azure AD users. It simply validates input received from its gRPC API and manipulates peers on the Wireguard interface.</p>
<p>A low privileged user runs <a target="_blank" href="https://github.com/JonCooperWorks/wireguardhttps">WireguardHTTPS</a>, a REST API that provides authentication, IP address allocation, device DNS settings, and device management. It is exposed directly to the internet without a reverse proxy. WireguardHTTPS allocates IP addresses in its database from a subnet provided by the user at setup. WireguardHTTPS assigns devices IP addresses from this pool, and a device’s IP address is released when it is deleted. It uses wgrpcd’s gRPC API to perform the desired operations on the Wireguard interface. PostgreSQL transactions ensure Wireguard interface and database are kept in sync to avoid IP address conflicts and stale data. WireguardHTTPS does not store any identifying information from users apart from their authentication provider ID, and can easily be modified to support other OpenID providers like AWS Cognito.</p>
<p>WireguardHTTPS is meant to serve its UI: <a target="_blank" href="https://github.com/JonCooperWorks/wgreact">wgreact</a>, a ReactJS single-page app. It uses <a target="_blank" href="https://github.com/gin-gonic/gin">Gin</a>’s static file server <a target="_blank" href="https://github.com/gin-contrib/static/">middleware</a> to serve the final Javascript application generated by webpack with a restrictive Content Security Policy, anti-framing headers, anti-XSS headers, and a limited set of TLSv1.2 and TLSv1.3 cipher suites. It is a single page and allows users to add, view, rekey, and remove their devices.</p>
<h1 id="use-cases">Use Cases</h1>
<ul>
<li><p>Expose services in private networks to connected devices, even if they’re behind a NAT or in a private VPC. You can host an instance of <a target="_blank" href="https://about.gitlab.com/install/">Gitlab</a> on a computer at home to create a private source code repository and much more.</p>
</li>
<li><p>Move unencrypted traffic from untrusted networks to cloud networks to protect against man in the middle attacks.</p>
</li>
<li><p>Use a static IP address from any network, even on a cell phone.</p>
</li>
</ul>
<h1 id="source-code">Source Code</h1>
<p>This entire setup is open source. You can see the source code for <a target="_blank" href="https://github.com/JonCooperWorks/wgrpcd">wgrpcd</a>, <a target="_blank" href="https://github.com/JonCooperWorks/wireguardhttps">WireguardHTTPS</a> and <a target="_blank" href="https://github.com/JonCooperWorks/wgreact">wgreact</a> on Github. WireguardHTTPS is still under development and has not yet been audited. I don’t recommend using this software for anything important. “WireGuard” and the “WireGuard” logo are registered trademarks of Jason A. Donenfeld.” You can download Wireguard at <a target="_blank" href="https://www.wireguard.com/">https://www.wireguard.com/</a></p>
]]></content:encoded></item><item><title><![CDATA[Automated API testing with Postman]]></title><description><![CDATA[Postman is an excellent API testing tool for developers, QA testers and penetration testers. Its UI allows you to easily send HTTP requests and see responses, but it’s also a great automation tool.



Getting stock prices from Alpha Vantage with Post...]]></description><link>https://joncooperworks.com/automated-api-testing-with-postman</link><guid isPermaLink="true">https://joncooperworks.com/automated-api-testing-with-postman</guid><dc:creator><![CDATA[Jonathan Cooper]]></dc:creator><pubDate>Fri, 19 Jul 2019 05:00:00 GMT</pubDate><content:encoded><![CDATA[<p><a target="_blank" href="https://www.getpostman.com/">Postman</a> is an excellent API testing tool for developers, QA testers and penetration testers. Its UI allows you to easily send HTTP requests and see responses, but it’s also a great automation tool.</p>
<img alt="Image for post" src="https://miro.medium.com/max/4576/1*RBfDiJbakuZsrG99-rUHog.png" />

<blockquote>
<p>Getting stock prices from <a target="_blank" href="https://www.alphavantage.co/">Alpha Vantage</a> with Postman</p>
</blockquote>
<p>Postman allows you to write <a target="_blank" href="http://www.chaijs.com/">chai.js</a> tests in Javascript that will run after each response and let you make assertions about the response body. These tests can also be run headlessly with <a target="_blank" href="https://www.getpostman.com/docs/v6/postman/collection_runs/command_line_integration_with_newman">Newman</a> and added to your build pipeline, but I’ll talk more about that later.</p>
<p>The Postman test Sandbox contains several useful libraries, functions and objects for testing. A list of them can be found on the Postman Sandbox <a target="_blank" href="https://www.getpostman.com/docs/v6/postman/scripts/postman_sandbox">reference page</a>.</p>
<h2 id="contract-tests">Contract Tests</h2>
<p>Contract tests allow you to verify that API request and response schemas have not changed. This is especially useful when working on a mobile application with a large team with separate mobile and API developers.</p>
<p>First you must generate a <a target="_blank" href="https://spacetelescope.github.io/understanding-json-schema/about.html">JSON schema</a> for the response bodies. If you don’t want to do this by hand, you can use an online generator, like the one at <a target="_blank" href="https://jsonschema.net/">jsonschema.net</a>. For this demonstration, I’ll be running tests that verify that Alpha Vantage’s Stock Quote API contract has not changed.</p>
<p>It is convenient to store schemas, base URLs and other common variables using Postman’s environment variables and parse the JSON in each test, to allow multiple tests to reuse the same schema. Postman’s Sandbox comes with <a target="_blank" href="https://github.com/geraintluff/tv4">tv4</a>, a JavaScript schema validation library, that handles all the heavy lifting. All that is required of you is to call the <a target="_blank" href="https://github.com/geraintluff/tv4/blob/master/README.md#usage-2-multi-threaded-validation"><em>tv4.validateResult</em></a> function.</p>
<img alt="Image for post" src="https://miro.medium.com/max/4516/1*VddsiJz3vrW7-oBNdQfbIQ.png" />

<blockquote>
<p>Testing that Alpha Vantage’s stock quote API contract has not changed.</p>
</blockquote>
<p>The function will return an object with a <em>valid</em> property that will be true if the object matches the schema.</p>
<p>In addition to the schema, it is useful to check that response codes and content types are all as expected. Since Postman does not return much detail about failures, it is best to only put one assertion per test, to pinpoint exactly where the error has occurred.</p>
<p>You can test this sample in Postman with the <a target="_blank" href="https://gist.github.com/JonCooperWorks/efe3fbb1dc861f7a3f0f50429548415c">collection</a>.</p>
<h2 id="security-tests">Security Tests</h2>
<p>Postman tests can also be used to perform automated security tests.</p>
<p>Combined with an automated build pipeline, security tests that fail when a vulnerability is present will prevent vulnerabilities from being re-introduced at a later date by developers.</p>
<p>Since jail is real, I’ll be demonstrating this suite using <a target="_blank" href="https://google-gruyere.appspot.com/">Google Gruyere</a>’s sandbox.</p>
<p>Sometimes real world exploit scenarios in web applications involve multiple steps. Many vulnerabilities require authentication tobe exploited. Thankfully, Postman makes this easy using its collections test runner.</p>
<p>In Google Gruyere, there is an authenticated stored XSS vulnerability where a user can submit JavaScript code in a snippet and have it executed by anyone who views their instance’s home page.</p>
<p>First, we need to log in and get the authentication cookie.</p>
<img alt="Image for post" src="https://miro.medium.com/max/4544/1*RWiqe34wPcxa-6IVwYXo7Q.png" />

<blockquote>
<p>Logging in to Google Gruyere using Postman and setting the cookie in the environment.</p>
</blockquote>
<p>Postman allows you to set environment variables by using the <em>pm.environment.set</em> function. This is very useful for test cases that are dependent on the responses of previous requests, such as authentication headers and account numbers. Thanks to this, tests can be more dynamic and cut down on the use of hardcoded data.</p>
<p>Since this application uses cookies, we set the cookie into the environment variable “cookie”.</p>
<img alt="Image for post" src="https://miro.medium.com/max/3056/1*iHQygT3Oa8Oj8aqRFsxabw.png" />

<blockquote>
<p>Environment variables are surrounded by 2 curly brackets.</p>
</blockquote>
<p>Cookies can be managed by setting the Cookie header to the “cookie” Postman environment variable. Doing this on each request will automatically include the correct value.</p>
<img alt="Image for post" src="https://miro.medium.com/max/4536/1*rt-85gqz7U1uNWdDAvwzjQ.png" />

<blockquote>
<p>Submitting XSS payload.</p>
</blockquote>
<p>Next, we submit the request with the payload. The input should be validated in this case to ensure it doesn’t contain dangerous HTML payloads while still allowing harmless HTML through.</p>
<img alt="Image for post" src="https://miro.medium.com/max/4532/1*8OazTbg4101VzCwSonw3Og.png" />

<blockquote>
<p>This test will fail if the XSS vulnerability is still present.</p>
</blockquote>
<p>Finally, we test that the unencoded JavaScript payload that triggers the vulnerability has been sanitized. Once the developers have fixed these bugs, this test suite will pass.</p>
<img alt="Image for post" src="https://miro.medium.com/max/4300/1*sOcffsj895JbdZsXdj_Jxg.png" />

<blockquote>
<p>The Postman test runner running through the exploit chain.</p>
</blockquote>
<p>Writing security tests using Postman allows for a test suite that will pass when vulnerabilities are fixed. It is important to note that a passing test suite does not mean that the vulnerability has been patched fully, so manual testing with other vectors is still necessary. Creating a test case to reproduce the additional vector will help the developers create more secure software.</p>
<p>If you want to play with this collection, you can download it <a target="_blank" href="https://gist.github.com/JonCooperWorks/2c6ec3af2d3864cf0e1240988ad9d218">here</a> and the environment <a target="_blank" href="https://gist.github.com/JonCooperWorks/07a06330a4bd792fe8e895f9c66d08fd">here</a>.</p>
<h2 id="newman-and-your-pipeline">Newman and your Pipeline</h2>
<p>To really see the benefits of automated Postman tests, it’s best to add them as a build step using your Continuous Integration build pipeline software. Newman allows you to run Postman test collections whenever code is pushed to your repository.</p>
<p>First, export both your environment and collection as JSON files. Then, simply tell Newman where your environment and collection are and run it.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="2b3bdfdf497731694433e232e0a2544f"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/2b3bdfdf497731694433e232e0a2544f" class="embed-card">https://gist.github.com/JonCooperWorks/2b3bdfdf497731694433e232e0a2544f</a></div><p>Adding that command to your build pipeline will let Newman run your tests every build, and prevent regressions.</p>
<h2 id="try-it-out">Try It Out</h2>
<p><a target="_blank" href="https://targetpractice.network">TargetPractice</a> has vulnerable servers that you can hack to your heart’s content. Test real tools and exploits that work on live targets without going to jail. It’s not a crime if it’s <a target="_blank" href="https://targetpractice.network">TargetPractice</a>.</p>
]]></content:encoded></item><item><title><![CDATA[Cloak and Dagger — Malware Techniques Demystified]]></title><description><![CDATA[The cloak and dagger attack exploits a combination of drawing over other apps and the high level of access to other apps given to accessibility services on Android. It is a simple yet effective technique being exploited in the wild today by cybercrim...]]></description><link>https://joncooperworks.com/cloak-and-dagger-malware-techniques-demystified</link><guid isPermaLink="true">https://joncooperworks.com/cloak-and-dagger-malware-techniques-demystified</guid><dc:creator><![CDATA[Jonathan Cooper]]></dc:creator><pubDate>Wed, 03 Apr 2019 05:00:00 GMT</pubDate><content:encoded><![CDATA[<p>The <a target="_blank" href="http://cloak-and-dagger.org/">cloak and dagger attack</a> exploits a combination of drawing over other apps and the high level of access to other apps given to <a target="_blank" href="https://developer.android.com/reference/android/accessibilityservice/AccessibilityService">accessibility services</a> on Android. It is a simple yet effective technique being <a target="_blank" href="https://securityaffairs.co/wordpress/83005/malware/android-trojan-gustuff.html">exploited in the wild</a> today by cybercriminals.</p>
<img alt="Image for post" src="https://miro.medium.com/max/1920/0*kFyVUiYiDHhbymGl.jpg" />

<blockquote>
<p>No, not like that</p>
</blockquote>
<h1 id="how-it-works">How it works</h1>
<p>The cloak and dagger attack takes advantage of two Android permissions:</p>
<ol>
<li>SYSTEM_ALERT_WINDOW (The Cloak)</li>
<li>BIND_ACCESSIBILITY_SERVICE (The Dagger)</li>
</ol>
<h1 id="the-cloak">The Cloak</h1>
<p>SYSTEM_ALERT_WINDOW is used to draw over other Android apps. If an app installed from the Play Store requests this permission in its AndroidManifest.xml file, it will automatically be granted by the system. This forms the first building block of the cloak part of the exploit. Many apps use this permission for legitimate purposes, like Google Maps.</p>
<img alt="Image for post" src="https://miro.medium.com/max/1000/1*rI9vKouWXwYTM_4dcjC1XA.png" />

<blockquote>
<p>Google Maps uses SYSTEM_ALERT_WINDOW to display directions when its in the background</p>
</blockquote>
<p>Malicious apps, however, can take advantage of this feature to cover a user’s UI and trick them into clicking activities below the overlay. In the cloak and dagger attack, the user is tricked into enabling the malware’s accessibility service.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="88551f03adb4c62b01041b131d970b96"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/88551f03adb4c62b01041b131d970b96" class="embed-card">https://gist.github.com/JonCooperWorks/88551f03adb4c62b01041b131d970b96</a></div><p>For security reasons, Android only shows the coordinates of taps made inside an overlay to an overlay’s <a target="_blank" href="https://developer.android.com/reference/android/view/View.OnTouchListener">OnTouchListener</a>. Any taps outside the overlay will return the coordinates (0, 0). While this hides the exact coordinates of touches outside the malicious app’s overlay, the app can cover the entire screen except the areas they want the user to click. Since Android returns (0, 0) for touches outside the screen, a malicious OnTouchListener can cover the screen and adjust the overlays to disguise the next stage of the attack once the user has clicked on the desired area.</p>
<img alt="Image for post" src="https://miro.medium.com/max/3000/1*lSewb-iJW4WHv2SE-Axf4g.png" />

<blockquote>
<p>The cloak portion of the attack. The entire screen is covered except the areas that trick the victim into enabling the accessibility service.</p>
</blockquote>
<p>The demonstration uses translucent overlays, but a real attack would use an opaque distraction.</p>
<h1 id="the-dagger">The Dagger</h1>
<p>Once the victim has enabled our accessibility service, we have de-facto control over the device. Android sends the service’s <a target="_blank" href="https://developer.android.com/reference/android/accessibilityservice/AccessibilityService#onAccessibilityEvent(android.view.accessibility.AccessibilityEvent">onAccessibilityEvent method</a>) information after most user activity, including keystrokes, lock screen key presses, URLs and much more.</p>
<img alt="Image for post" src="https://miro.medium.com/max/3928/1*c7r3IHfFAFMXGw-ZCcswVg.png" />

<blockquote>
<p>The dagger capturing a user’s lock screen PIN</p>
</blockquote>
<p>The accessibility service is also able to send touch events to other apps, allowing it to take control of the device. At this point, we don’t even need root. The cloak fun also doesn’t have to stop with tricking the user into enabling the accessibility service. Since the accessibility service gets notified of a victim’s every move, malware can use overlays to hijack a victim’s banking apps and hide evidence of itself in the accessibility settings menu.</p>
<h1 id="demo">Demo</h1>
<p>A proof of concept is on <a target="_blank" href="https://github.com/JonCooperWorks/implant/">Github</a>. You’ll have to <a target="_blank" href="https://lastpass.com/support.php?cmd=showfaq&amp;id=9932">manually enable</a> the “Draw Over Other Apps” permission since you’ll be sideloading this app. The attack has been tested on the following devices:</p>
<ul>
<li>Nexus 5X Nougat</li>
<li>Nexus 5X Marshmallow</li>
<li>Nexus 4 Lollipop</li>
</ul>
<p>It’s best to use an emulator of one of those images. You’ll have to adjust the overlay sizes in the Stage subclasses to match the screen size of any other device you want to get it working on.</p>
<h1 id="level-up-your-hacking-skills">Level Up Your Hacking Skills</h1>
<p><a target="_blank" href="https://targetpractice.network/signup/?next=/">TargetPractice</a> has vulnerable servers that you can hack to your heart’s content. Test real tools and exploits that work on live targets without going to prison. It’s not a crime if it’s <a target="_blank" href="https://targetpractice.network/signup/?next=/">TargetPractice</a>.</p>
]]></content:encoded></item><item><title><![CDATA[Disabling OkHttp’s SSL Pinning on Android Apps]]></title><description><![CDATA[Your target has an Android application and you want to walk through their API to check for server-side vulnerabilities. You configure the emulator to use Burp Suite as a proxy and begin using the app.
https://gist.github.com/JonCooperWorks/1866d3efe7...]]></description><link>https://joncooperworks.com/disabling-okhttps-ssl-pinning-on-android-apps</link><guid isPermaLink="true">https://joncooperworks.com/disabling-okhttps-ssl-pinning-on-android-apps</guid><category><![CDATA[Android]]></category><category><![CDATA[SSL]]></category><dc:creator><![CDATA[Jonathan Cooper]]></dc:creator><pubDate>Thu, 27 Sep 2018 05:00:00 GMT</pubDate><content:encoded><![CDATA[<p>Your target has an Android application and you want to walk through their API to check for server-side vulnerabilities. You configure the emulator to use <a target="_blank" href="https://portswigger.net/burp">Burp Suite</a> as a proxy and begin using the app.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="1866d3efe7cce03a7efa11d26633bc31"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/1866d3efe7cce03a7efa11d26633bc31" class="embed-card">https://gist.github.com/JonCooperWorks/1866d3efe7cce03a7efa11d26633bc31</a></div><p>Suddenly, the app stops working. Nothing shows in Burp and no HTTPS requests work. The developers have implemented <a target="_blank" href="https://www.owasp.org/index.php/Certificate_and_Public_Key_Pinning">SSL pinning</a> and your phony certificate has been detected. Fortunately, SSL pinning can be disabled if you’re willing to get your hands dirty.</p>
<h2 id="decompiling-the-app">Decompiling the App</h2>
<p>First, you need to decompile the app. <a target="_blank" href="https://ibotpeaches.github.io/Apktool/">Apktool</a> works great for this, and it’s available on all platforms. Follow the <a target="_blank" href="https://ibotpeaches.github.io/Apktool/install/">install instructions</a> and then decompile the APK.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="227186e621068678c665d4a6634fa24f"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/227186e621068678c665d4a6634fa24f" class="embed-card">https://gist.github.com/JonCooperWorks/227186e621068678c665d4a6634fa24f</a></div><p>This command will produce a directory with the AndroidManifest.xml, resource files and <a target="_blank" href="https://source.android.com/devices/tech/dalvik/dalvik-bytecode">Smali bytecode</a>.</p>
<h2 id="removing-the-pin">Removing the pin</h2>
<p>Next, you’ll need to remove the pinned certificate from the application. It’s easiest to use grep to look for “CertificatePinner”.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="303eb5e45916a7390828feb52c681e07"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/303eb5e45916a7390828feb52c681e07" class="embed-card">https://gist.github.com/JonCooperWorks/303eb5e45916a7390828feb52c681e07</a></div><p>This will return a list of files in the app that use SSL pinning with OkHttp. It will bring up instances in third party libraries that may need to be disabled as well. Look through to see where the app is pinning its certificate. Once you find it, open the file in your favourite text editor. According to OkHttp’s <a target="_blank" href="https://square.github.io/okhttp/3.x/okhttp/okhttp3/CertificatePinner.html">CertificatePinner documentation</a>, certificate hashes are added using the CertificatePinner.Builder’s add method. We need to look for the Smali bytecode that corresponds with the method call and remove it to neuter the SSL pinning.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="7716c77d6a5086291cd8afb741c0372f"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/7716c77d6a5086291cd8afb741c0372f" class="embed-card">https://gist.github.com/JonCooperWorks/7716c77d6a5086291cd8afb741c0372f</a></div><p>Removing the two lines above will get rid of a pinned certificate. You’ll have to repeat this for every certificate hash the app pins.</p>
<h2 id="rebuilding-the-apk">Rebuilding the APK</h2>
<p>After you’ve removed the SSL pinning, rebuild the APK using apktool. You’ll have to <a target="_blank" href="https://developer.android.com/studio/command-line/zipalign">zipalign</a> the APK and resign it with your signing key to get Android to accept it.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="cf4666eff44b64b51f5a42d2be269d45"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/cf4666eff44b64b51f5a42d2be269d45" class="embed-card">https://gist.github.com/JonCooperWorks/cf4666eff44b64b51f5a42d2be269d45</a></div><p>If you don’t have a signing key, you can generate one using keytool.</p>
<div class="gist-block embed-wrapper" data-gist-show-loading="false" data-id="dbaf9ac6ed3197f091af433522833a69"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a href="https://gist.github.com/JonCooperWorks/dbaf9ac6ed3197f091af433522833a69" class="embed-card">https://gist.github.com/JonCooperWorks/dbaf9ac6ed3197f091af433522833a69</a></div><p>Finally, install the APK on the target device. If the previous version is already installed, you’ll have to uninstall it so Android doesn’t detect the different signatures.</p>
<p>After doing this, you should be able to intercept requests from the app.</p>
<p>Happy hacking!</p>
]]></content:encoded></item></channel></rss>