Friday, 13 October 2017

Simple ways to add and work with a `.jar` file in your local maven setup

TL;DR Hack - add as a library in IntelliJ project. Tactic - add as system scope in maven. Tactic/Strategic - install locally to .m2. Strategic - use a repository management tool, publish to maven central

Sometimes you want to work with a jar file that isn’t hosted in maven central.

It might be a 3rd party jar, it might be one that you have written.


You have a lot of options for this. The approaches that I have used:

  • add .jar files as an IntelliJ project dependency
  • install it locally to your .m2 repository
  • add it to your project as a system scoped file
  • use a repository management tool: like Nexus or Archiva
  • publish the dependency to maven central

Thursday, 12 October 2017

How to Diff Java Code in IntelliJ - 3 ways to use the Compare Tool

TL;DR IntelliJ has an inbuilt diff tool which you can use to compare files, classes or code with the clipboard. Just right click and choose Compare.

I was busy refactoring code in RestMud this morning because I want to try and open source the basic game engine, and then later the Web/REST API, but I want to tidy up the code a little first.

I’ve been working through the code:

  • moving classes into new packages
  • splitting classes to make code clearer and easier to manage
  • new classes make code easier to test

And the more I tidy it up in general, the easier it is to spot smaller problems of code duplication.

Thursday, 5 October 2017

Java 1.9 Reflection and Package Access Changes

TL;DR Java 1.9 does not like reflection as much as previous versions and access to proprietary packages is not possible out of the box.

Java 1.9 is out, and many companies now will be evaluating Java 1.9 and migrating their code base over to Java 1.9. I am now going through a similar process.

I have some fairly simple code that I was using for HTTP requests. This was originally written in Java 1.8 and to keep my codebase as simple as possible and I wanted to use no external libraries, therefore for HTTP requests I used an HttpURLConnection.

This started to fail on Java 1.9.

When I looked in my code more carefully I realised that it wasn’t HttpURLConnection that was failing in Java 1.9 it was my use of Reflection to bypass some constraints of the HttpUrlConnection that was failing.

What contraints?

With the HttpUrlConnection I could not find a way to get the actual request headers being sent.

I could see that the URLConnection class does have all the request headers:

private MessageHeader requests;

But it keeps them all to itself.

The class does support a mechanism to return these with the getRequestProperties method, which is public.

public Map<String, List<String>> getRequestProperties() {


But, wait a minute.

public Map<String, List<String>> getRequestProperties() {
return this.requests == null?Collections.emptyMap():this.requests.getHeaders((String[])null);

getRequestProperties does not actually want to return them if we are connected, the checkConnected method throws an exception if we try.

Fair enough.

I debug the code and find out when we can get them, and I’ll take them after they are set, but before we are connected.

But I could not actually find that point.

int statusCode = con.getResponseCode();

Prior to the above line I am not connected, and there are no request headers accessible.

After the above line, the request has been made, I am connected and the headers are not accessible.

I tried disconnecting, and a whole bunch of stuff, but since I had a problem to solve and the ‘class’ was getting in my way, I turned to Reflection to solve it.

What I did that failed

Since I new that the requests field on the URLConnection had the information that I needed, I simply used reflection to go and get the information that I needed.

Field field = con.getClass().getDeclaredField("requests");
MessageHeader requestHeaders = (MessageHeader) field.get((URLConnection) con);

lastRequest = new HttpRequestDetails();

int keyIndex = 0;
while(keyIndex < requestHeaders.getHeaders().size()){
lastRequest.addHeader(requestHeaders.getKey(keyIndex), requestHeaders.getValue(keyIndex));

And all of that was very jolly, until Java 1.9 came along.

Migrating to Java 1.9

After debugging the problem I faced, I found the following blog post from provided useful information:

The problem I faced was that when I migrated to Java 1.9 I faced the issue of:

Error:(7, 19) java: package does not exist
Error:(169, 13) java: cannot find symbol
  symbol:   class MessageHeader
  location: class com.javafortesters.course.casestudy.http.HttpRequestSender
Error:(169, 45) java: cannot find symbol
  symbol:   class MessageHeader
  location: class com.javafortesters.course.casestudy.http.HttpRequestSender

Now I found this odd.

The import of wasn’t throwing any errors. Until compilation.


And when I looked in the code for URLConnection the responses field was still there and was still of of type

So the package does exist, but just not for me.

Java 1.9 has finally made proprietary packages inaccessible. This was a long time warning in Java and has now come to fulfillment.

It is still in the JRE, but not accessible to me due to Java 1.9 changes.

Very well then.

Since the getRequestProperties method still exists, I decided to bypass the check for this.checkConnected();

I’m sure the library probably has very good reasons for not wanting me to access the List while we are connected, but in my code I view this as low risk since I have a single thread, I’ve made the request, I just want to document the headers that were sent.

So I used reflection to toggle the connectedfield value so that the checkConnected() method would not throw an exception.

Field connected = con.getClass().getSuperclass().getSuperclass().getDeclaredField("connected");
connected.setBoolean((URLConnection) con, false);

Map<String, List<String>> requestHeaders = con.getRequestProperties();
for(String header : requestHeaders.keySet()){
    StringBuilder headerValue = new StringBuilder();
    for(String headerText : requestHeaders.get(header)){
    lastRequest.addHeader(header, headerValue.toString());
connected.setBoolean((URLConnection) con, true);

And this worked in Java 1.9, and 1.8 and 1.7

But… Java 1.9 Doesn’t like reflection as much

But Java 1.9 tells me that this reflective access operation is ILLEGAL!

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.javafortesters.course.casestudy.http.HttpRequestSender
 (file:/) to field
WARNING: Please consider reporting this to the maintainers of com.javafortesters.course.casestudy.http.HttpRequestSender
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

I assume this is due to the new Module system in Java 1.9

So while my code works, it will probably fail again in the near future.

Reflection Comes with Risk

Reflection always comes with the Risk that the underlying code you are using changes.

And it has every right to do so.

I should not be accessing its private fields and methods.

But sometimes, when testing, I need access to features and functionality that, for seemingly arbitrary reasons, the class does not support.

Had this library been written to fully support a ‘testing’ process, where we don’t just want to make requests, we want to know exactly what details were sent in that request, then I wouldn’t have this problem.

The reason I learned to use reflection in Java in the first place was because I’ve worked with ‘frameworks’ that do stuff, that I want to use as part of my test process to validate results, or to trigger situations that are necessary for testing, but which the ‘framework’ decided that no-one would want to do that.

In theory, there is nothing wrong with the URLConnection telling me what the request headers are. It could have added them to a Synchronised or immutable collection, it is pure documentation.

Sometimes good solid technical reasons for functionality gets in the way of testing.


I could have added another HTTP library to my code to make the requests and give me access to the raw data.

I could have added a code based HTTP Proxy, sent all requests through the proxy and used the proxy to gain access to the headers.

All of that was possible, but went against my strategic aims of keeping the code base as free of external dependencies as possible, so I adopted a tactical maneuver to gain access to what I needed.

Final Results

In the public release of the above code I have removed all the reflection. But I have kept it in the code I use for internal testing because I can control the JVM and settings that the code runs against.

I will continue to use reflection to bypass constraints in libraries and frameworks that I use as a tactical measure to get work done.

Relying on this approach for strategic work is a risk that might bite you, at the time when you are least flexible to respond, so care needs to be taken when you do this.