2

I figured this was a good place to learn a little about trusting libraries such as HTMLUnit with sensitive information.

I want to build a service that categorises my spending, kind of like my own personal mint.com. Can I trust an open source library such as HTMLUnit with passwords to sign into my bank account and scrape data. Additionally I would like to know the most secure way to implement the dependency. Currently the dependency is downloaded into my Play framework project via maven. To do this all I had to do was add the following line to the build.sbt file:

libraryDependencies += "net.sourceforge.htmlunit" % "htmlunit % "2.20""  the build.sbt"

Is this a secure way to use such a dependency? Or is there a better way to add the dependency to my project?

Any advice would would be greatly appreciated, thanks in advance for your help.

DominoSug
  • 31
  • 3
  • As long as it runs on your own computer it should be fine - Java isn't really vulnerable to buffer overflows or similar exploits, so worst case scenario your app would just not work as expected but wouldn't execute malicious code. Just be careful when dealing with TLS to make sure you're talking to your bank and not someone else. – André Borie Mar 17 '16 at 18:25
  • Any http requests to my bank are rerouted to https automatically, http requests dont work, so it appears the only way I can use my internet banking is via TLS. Is there anything else I can do on my end to ensure that TLS is used? – DominoSug Mar 18 '16 at 10:23
  • Also sorry, why do you say "as long as it runs on your own computer?", What threats are you concerned about if I run this on another computer (i.e. running this from a secure web server) – DominoSug Mar 18 '16 at 10:27
  • The bank's legitimate server is redirecting HTTP to HTTPS. If it was an attacker he definitely wouldn't do that as it's in their best interest if you continue to use HTTP. If you run this on a server you should make sure your server isn't compromised. – André Borie Jul 16 '16 at 00:36
  • For prior art, take a look at the OFX protocol, and GnuCash's documentation of same. It is possible, though unlikely, that your specific bank still offers this mechanism to retrieve your transaction records, saving you from screen scraping. If/when screen scraping, follow Noir's advice and be very careful about "testing" against your bank's servers. Many banks now monitor for this behavior. – Jonah Benton Aug 15 '16 at 05:41

1 Answers1

1

Best practice in software testing is to avoid testing with production systems (Your banking site is a production system and additionally I bet automated crawling is a non intended use case) and further to not test with real data.

I'd try to mock most output of the banking site until I'm pretty sure my code is working as intended. The tests could be interpreted as a hacking attempt which could result in a temporary ban.

However I don't see an issue with using HTMLUnit since it's pretty transparent what the library is doing.

For the second part: Make sure to get the dependency via TLS.

Noir
  • 2,523
  • 13
  • 23
  • thanks for your response, I am using plays dependancy management to download HTMLunit. The Play Framework uses SBT which in turn uses the existing Maven Central repository. the Libraries I use are downloaded when I build the project. How can I ensure that this is using TLS? – DominoSug Mar 18 '16 at 10:32