No Fluff Just Stuff Austin 2014

I went to my favorite software conference again: No Fluff Just Stuff. The main reasons I attend are the great speakers and small class sizes provide for the opportunities to have real-world conversations about my environment.

The good and bad part of this conference is that it is on a weekend. It has little impact on my working hours, but I do lose a weekend. Most of the time my family will come with me and we’ll make an adventure out of it, but unfortunately schedules didn’t line up this year.

The fact that it is on a weekend is why many people from work probably do not want to attend. I cannot blame them, but they would gain huge benefits from talking with people who are already well experienced in where we want to be.

I have years of notes from NFJS, but putting them into soft copy always gets pushed to a back burner. Not this year!

Technology Radar – Neal Ford

this is one of my favorite talks every year, but unfortunately I had a conflicting session and could not attend. I put it here mainly for reference.

Key technologies/concepts to research

Continuous Delivery p1 – Neal Ford

  • Separate release from deployments
  • DevOps Kata – Single Line of Code – devopsy
    • how hard is it for your shop to deploy a one-line code change?
    • Dave Thomas’ CI book
  • Continuous Integration
    • check-in early and often
    • integration with other pieces is toughest part, and used to be done at end of project, which made it very difficult. You will find issues sooner, and integration will be much easier if done earlier in software life cycle
  • Continuous Deployment
    • Cycle Time – competitive advantage to try things ASAP and get immediate results instead of releasing once every 6 months and hope your users like the massive change
  • Scrummer-Fall – waterfall agile – very bad
  • Cruise Go – now-free build offering from Thought Works

Understanding Java Memory – Ken Sipe

Top Agile Pitfalls That Lead to Trouble – Andy Painter

  • Estimates are guesses and the sooner everyone in your organization realizes this, the happier everyone will be
  • Goal is to be a better software shop 1 year from today
  • Dr. Dobb’s agile surveys over the years on Agile adoption show huge trend up in usage
  • 2 pizza rule: no team should be so big that 2 pizzas cannot feed them
  • keep teams consistent. personnel changes drastically hurt the team’s performance since they have already experienced the Deming Cycle together
  • Key Agile Metrics
    • Velocity
      • measures how many user stories you average in a given time frame or sprint
      • poorly named: Planning Tool instead of accelerator.
      • Which is faster, waterfall or agile?
        • Apples & Oranges comparison trying to compare the two since their approach is so different
      • average of last 8 releases is good indication of velocity because in that time frame, people will have taken vacations, sick leave, etc.
      • after 3-4 sprints you will have a good indication of your potential for velocity
      • 30 pt velocity per week and over 6 weeks, you can do 180 stories. If Biz wants 200, then tough shit. Tough shit conversations are extremely important for the process, and forces everyone to make decisions on priorities sooner in the process.
      • possible to re-estimate stories as you go since you will have more knowledge of the system
    • Backlog
      • one and only one location for backlog
      • clear priority of backlog
      • enhances self-organizing dynamic of IT shop

Continuous Delivery for Architects – Neal Ford

  • If This Then That – IFTTT
  • Conway’s law
    • organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations
  • Book: Domain Driven Design by Vaugn Vernon
  • Environment Config/Build tools: Ancible, Docker, Puppet, Chef
  • Death by Coupling
    • Coupling is biggest issue, which means a simple change could have huge side effects/impact
  • 2 Pizza rule: teams must be small enough to be fed by 2 pizzas
  • Metrics & Monitoring
    • Cyclomatic complexity
      • measures function/method complexity by looking at logic and giving it a value
      • the higher the number, the more complex the method. code should be made of many, less complex methods instead of fewer large onces
    • Afferent Coupling – incoming references to method
    • Efferent Coupling – outgoing references to method
  • Tools
  • Tool Limitations
    • signal to noise
    • no 1 true metric to grade upon
    • gathered, but ignored
    • inaction/overaction
    • we should prefer trends over discrete values
  • Separate Feature from Release
  • Antipatterns – circular dependencies
  • Jigsaw — Project Kenai & OSGi
  • Tools to help prevent circular dependencies
  • Prefer REST over SOAP
    • versioning
      • prefer semantic over endpoint version
      • expand & contract paradigm, borrowed from DB structures
  • Consumer Driven Contracts
    • Created by Martin Fowler
    • allows endpoints to evolve w/ Consumers and provide tests
    • many consumers = successful product
  • (Micro)Services
    • monoliths vs. microservices
    • smart endpoints, dumb pipes
    • as decoupled and cohesive as possible
    • small, single responsibility
      • makes it easy to prefer rewrite over maintain
    • services should be JSON pumps for UI/Services to consume
  • Products, NOT Projects. You build it, you maintain it
  • Truck Number
    • Or more commonly called Bus factor in America
      • High bus numbers are good (with the best case being equal to the number of developers on a project). This means all developers understand the codebase more or less equally. A low bus number (especially, one) represents a high risk.
  • 2 phase commit is a really bad idea
    • banks existed before corporations
    • pony existed before ATM. Ponies did not have 2 phases
  • Automate synthetic transactions to smoke test deployments
  • Tool
    • Dropwizard – Template service similar to Jersey
  • Video: Micro Services: Java, the Unix Way
  • Components are deployed. Features are released.
    • use toggle to enable/disable features
  • Dietzler’s Law – users want 100% of what they want and nothing less
    • 80-10-10 Rule: 80% is doable in a relatively short amount of time, 10% is possible, but lots of effort, and the last 10% is impossible due to technical restrictions
  • Software does not age, it rots
  • Tech Debt
    • defects vs. debt
      • defects are bugs in the code
      • debt are bugs in how it was coded, whether it causes a bug or not
  • Yesterday’s best practice is today’s anti-pattern

Android p1 – Ken Kousen

  • Android Studio (AS) for dev is prefered over Eclipse plugin
    • Google makes updates to AS, but not Eclipse lately
  • Android Developers
  • Dalvik is being switched for ART (Android RunTime)
  • Testing must be done on device or emulator
    • de facto integration test only
    • will prob change to Unit test features in future
  • will move to Gradle builds in near future
  • OS aspects are all services
    • provides updates so carriers cannot pigeon hole users to a specific version
    • Forks – lose all services when forking codebase
      • only two successful forks: Kindle and the major flavor in China who wrote their version of all services
  • Market share: Android ~80%
  • Revenue share: Apple slightly higher than 50%
  • Generate code for both
  • Hello World example
  • package name is very important for Google Play Store to distinguish apps
  • XML project config is not deployed w/ app
  • XML is what generates the
  • built in localization via /res/strings.xml
  • Apps live on callbacks
  • Manifest file is starting point to see entry into app
  • Linear Layout vs. Relative Layout
    • default for Linear is horizontal, but prob want vertical for form data
  • EditText
    • like HTML textfield
  • @+id – is for defining ID
  • @id – is for referencing ID
  • Views are widgets and can be groups of Views
  • AVD Manager
    • emulator
    • HAX Plugin and Seperate install for enhanced emulator performance (seconds vs. minutes)
  • Minimum and Target SDK
    • apps are configured for a minimum compatibility and a target compatibility of an SDK

Android p2 – Ken Kousen

  • Compile with – pick latest because it will handle compatibility plumbing
  • Fragmentation will get better over time
  • Higher res photos and art will scale better on all devices
  • Eclipse test requires 2nd project w/ dependency on primary project
    • hard to test outside of IDE (e.g. during build)
    • gradle will fix this eventually
  • Eclipse has WYSIWYG editor that is sometimes easier to deal with
  • Activity Lifecycle
    • 1 screen is an Activity
  • Logging
    • not like log4j – all methods static
    • log.v(TAG, “msg”)
    • = What a Terrible Failure
    • Filters – strings
    • Tag – to filter when viewing
  • Android Debug Bridge
    • adb devices
    • kill-server
    • start-server
  • config – always prompt to pick device (e.g. emulator, or attached phone/tablet)
  • Favorite apps only call certain Lifecycle events
  • roate device calls certain life cycles
    • data must be saved and account for this, but many times actions occur so fast it’s not an issue
  • onSaveInstantState()
    • save state of app
    • only happens if doDestroy
    • outstate.putStr(display)
    • restored in onCreate()
  • Providing Resources
    • Nighte Mode
  • Android lint
    • The Android lint tool is a static code analysis tool that checks your Android project source files for potential bugs and optimization improvements for correctness, security, performance, usability, accessibility, and internationalization.
  • emulator can leverage camer on computer
    • Broadcast-Recevers – emulate phone actions
      • text message, GPS, incoming call, battery failure
  • SensorSimulator
    • for simulating sensor data in real time
  • Google Play requires 2 screenshots to be in Google Play Store
  • Droid @ Screen
    • Show your Android device for your audience
  • Toast
    • little pop-up that fades away with useful info
    • toast.makeText(…).show()
    • can customize screen location with gravity
  • Intent
    • object trying to transfer data from A to B
    • Explicit & Implicity
      • Excplicit – Specify component to stage
      • Declare action to perform (in Manifest)
    • Extras
      • data carried to destination (key-value map)
  • Views & Adapters
    – lots of display options (e.g. layout for tables, etc.)
    • hello world
    • ICNDB project
  • Commonsware
  • The Busy Coder’s Guide to Android Development
    • $$$ – online book that is updated every 6 weeks or so

Android p3 – Ken Kousen

  • XML Layout is just starting point then programmatically change it
  • gradle can build variances of app (e.g. free, paid, etc.)
  • /gen = generated java files
  • issue: if file doesn’t generate, probably an XML file error
  • AsyncTask
    • prefered over Threads & Process in native Java
    • do NOT access UI thread outside of UI thread
    • use Concurrent & Execution instead of Thread class in Java
    • long running tasks, use Service object
    • users like to see progress bar
    • possible to disable rotation for longer executions, but users still may hit back button
    • Steps for AsyncTask
      • onPreExecute()
      • doInBackground()
      • onProgressUpdate()
      • onPostExecute()
    • cancel(boolean) to cancel task
    • resume download if issues – 3rd party libraries handle this better
      • probably included in future releases
    • IntentService – for long running tasks in background, not typically for GUI
  • MenuItem
  • ActionBar
    • split action bar
  • REST
    • org.json
    • android.util
      • JSonReader
      • JSonWriter
    • Spring for Android
      • RestTemplate – good rest client for general Java
      • wrap/unwrap JSON to objects and vice-versa
    • eclipse – must download file and put jars in /lib directory
      • if using gradle, it will automatically be done
    • GSon to convert back and forth between JSON and Objects
    • -> API
    • Check network connection
      • ConnectivityManager
      • getSystemService(Context.connectivityService)
      • NetworkInfo.isConnected()

My new favorite test tool: Unitils

I was working on some RESTful integration tests, and kept being annoyed at the repetetive nature in which I was verifying my user object. I knew I could override the equals() and hash() methods and simply call user.equal(newUser) to confirm whether users were equal, but I wanted a more universal solution that would not be limited to just my User objects. I knew someone had to have fixed this problem using reflection.

A few google searches later revealed the incredibly useful and powerful

Old, ugly, repetitive assertions

import org.apache.commons.lang3.StringUtils;
import org.junit.Assert;
import org.junit.Test;
import org.unitils.reflectionassert.ReflectionAssert;
import org.unitils.reflectionassert.ReflectionComparatorMode;

public class TestUserRestService {
    public void test_createUser() throws Exception {
        String xml = this.getResponseAsString(response.getEntityInputStream());

        User uFromXml = (User) WSUtils.getObjectFromXml(xml); 

        Assert.assertEquals(u.getUserId(), uFromXml.getUserId()); 
        Assert.assertEquals(u.getPassword(), uFromXml.getPassword()); 
        Assert.assertEquals(u.getFirstName(), uFromXml.getFirstName()); 
        Assert.assertEquals(u.getLastName(), uFromXml.getLastName()); 
        Assert.assertEquals(u.getPhone(), uFromXml.getPhone()); 
        Assert.assertEquals(u.getZip(), uFromXml.getZip()); 
        Assert.assertEquals(u.getEmail(), uFromXml.getEmail()); 
        Assert.assertEquals(u.getAddress(), uFromXml.getAddress()); 
        Assert.assertEquals(u.getCity(), uFromXml.getCity()); 
        Assert.assertEquals(u.getState(), uFromXml.getState()); 
        Assert.assertEquals(u.getZip(), uFromXml.getZip()); 

Simple, elegant one-line to verify code

        String xml = this.getResponseAsString(response.getEntityInputStream());

        User uFromXml = (User) WSUtils.getObjectFromXml(xml); 

        ReflectionAssert.assertReflectionEquals(u, uFromXml); 

That one line of code just replaced 12, and those 12 were not all attributes on the class, so that one line is much more useful than just replacing the initial 12 assertions mentioned above.

There were some issues with default values, but they have already thought of that and provide a parameter to ignore defaults.

        ReflectionAssert.assertReflectionEquals(u, uFromXml, ReflectionComparatorMode.IGNORE_DEFAULTS); 

I was kind of curious how it would handle certain Collection classes, and so far, for my simple Lists it appears to work as advertised.

    public static void main(String[] args) {
        List<String> g1 = new ArrayList<String>(); 

        List<String> g2 = new ArrayList<String>(); 

        User u1 = TestData.getTestUser_1("bob"); 

        User u2 = TestData.getTestUser_1("bob");

        ReflectionAssert.assertReflectionEquals(u1, u2); 

This utility has significantly improved my test classes by making them much more thorough, and concise. I may even look to enhance (I want to pass a list of attributes to ignore). I just wish it was on GitHub instead of Sourceforge.

Unix inode table max

We have a legacy application that creates a crap-load of files to track key data points within the app. Why is this not in a DB, I have no idea. Probably because the app was written in the 90’s, and it is so fragile and complex no one wants to touch it.

We recently migrated this app to an upgraded WebSphere environment, and OS, which took 15 different people from 5 different teams and a few months worth of effort. In this process, we knew there would be hiccups, and during one of my recent on-calls, I received a call with the error below:

19:12:22.763 [Thread-4795] ERROR - Database creation Error: /mount/data.xml (No space left on device)
          at Method) ~[na:6.0]
          at<init>( ~[na:6.0]
          at<init>( ~[na:6.0]
          at$ ~[classes/:na]
          at [na:6.0]
19:16:12.294 [WebContainer : 9] ERROR com.web.servlet.FileUpload - Error processing file upload:
19:16:12.295 [WebContainer : 9] ERROR com.web.servlet.FileUpload - java.lang.Exception: Error creating server directory
java.lang.Exception: Error creating server directory
          at java.lang.Throwable.<init>( ~[na:6.0]
          at [classes/:na]
          at [classes/:na]
          at javax.servlet.http.HttpServlet.service( [javax.j2ee.servlet.jar:na]
          at javax.servlet.http.HttpServlet.service( [javax.j2ee.servlet.jar:na]
          at []
          at []
          at []

“No space left on device” made me think that the mount was out of space, but a “df -h” command revealed that the mount had plenty of space. So is the error message completely bogus, or possibly related to the hand-held device that caused the error? We knew that orders were failing to be sent so there was an error somewhere in the app. After confirmation from developers that the hand-held devices were indeed not the space culprit, we got our Unix team on the horn, and they immediately knew the problem: the inode table had been maxed out.

His response is much more informative and elegant than I could have put together for a Unix topic:

The parameter changed on the filer was ‘maxfiles’, a setting which most applications never get close to maximizing. As the parameter name implies, it simply controls how many files exist at a given time on a volume and is enforced at a storage level from the NetApp rather than a filesystem level by the OS. When the problem occurred, the 3 million inode limit was reached and by nature of how it works could not be reduced until we first had a bit of overhead for breathing room. What we saw initially after increasing the limit was that 20,000 new inode assignments were made, but then when I checked a couple hours later it had dropped about 250,000. It has been fairly stable at about 2.82M now for the rest of the weekend. Given the drop, I would say we could likely reduce our ceiling as well down to perhaps 3.2M if we wanted to pull the reins in a little bit from the increase.

I bet somewhere there were Linux logs that referenced the inode issues, but unfortunately I didn’t have access to them (Splunk anyone?). It would have been nice if somewhere the word “inode” would have been used in the error message. It would have saved me and a few of my teammates an hour or two on a weekend.

AppDynamics License Issues


We renewed our AppDynamics license, and received the new license.lic file in an email like we always have. I FTP’d it over to our Controller, restarted it, and hit the home page, but it appeared that there was an issue:

I did the usual troubleshooting of restarting again (same result) and reverting to the original license.lic, but it failed too since the original license was out of date by a day. I opened a ticket with their support. We were using an on-premise and the latest version is 3.8.4. which was an issue since their license structure had drastically changed in one of the recent releases, causing this issue. Instead of trying to have them generate a new license for the the older Controller, I just upgraded to the latest Controller version: 3.8.4.

However, after the upgrade, we still had the same issue. Their support had me run the commands below, and send them the output.

/appd_home/bin/ login-db
SELECT * FROM accountG
/sbin/ifconfig -a

From this output, they were able to see that the machine’s MAC address had changed from what was originally on the AppD license. I followed up with our Unix team to ask if anything had changed on the machine’s VM, but nothing as far as they could see. So given this info, I could have had the Unix team change the MAC address, or just have AppD generate a new license with the new MAC address. We elected for AppD to generate a new license with the new MAC address since this was by far the least invasive option in our environment.

Hope this helps someone else going through an AppDynamics upgrade or license renewal from an older version.

Readers’ High

Last year, I read 20 books. This year, I have only read a few so far, but I have still been reading a significant amount of words, they just are not in books. I have been utilizing Feedly and Pocket to read a lot more in the technical realm, and longer articles.

Aside: I wish my Kindle, Feedly, and Pocket each provided some more statistics on how many articles I’ve read, and how long each one took me to read. Their plugins could then tell me how many words per minute I read, and how long a potential book/article would take me to read.

I have a few books I’ve been wanting to read for a while, and elected to start one last week: A Higher Call: An Incredible True Story of Combat and Chivalry in the War-Torn Skies of World War II

The book was fantastic, but I begin to hate myself when I get into a good book: I turn into an addict. I get a readers’ high.

I routinely went to bed well beyond midnight, which is against my initiative to maximize my productive time. I become very single-threaded in my reading efforts, and essentially bailed on all other research topics to see how this book ended as quickly as possible. I’m not sure if this is credit to the author, or a knock on myself. Either way, I was not nearly as productive on my personal areas of interest.

Now that I am done with the book, I hope to wrap up a few interesting technical ideas I’ve been wanting to research and post on. Or I’ll just start another book.