Autor: jakob

Kategorien
Book Coding Culture

Thoughts on Clean Agile from Robert C. Martin

My third book review leads me to the third book in a row from Robert C. Martin. You may think I am a fanboy or such, but its just that as written before all those books and more I got provided from my employer, and somehow I liked the flow of Martin’s books, so I read them one after the other. But fear not, the next book will be from another author.

So „Clean Agile“: Already the title is a bit triggering. I remember a online talk from another big agile representative (forget the name though) who was very clearly discouraging to use „Agile“ as a noun for anything, but always use it in its grammatically correct way as an adjective. Thats what I thought about when holding the book in my hand. However, I dont really care for the difference. Grammar is the one side, the other is that agile/agility has been a standing term in the sw industry for a while now, so one may as well use it in its noun form. Nevermind.

The first thing noteworthy is the history lesson about the waterfall process. According to Martin, the 1970 paper from Winston Royce which according to popular belief introduced waterfall and initiated its establishment, was actually arguing against this model. If that is true, a lot of people must have read only the first part of that paper and then went on implementing that part – because it chimed well with the notions of Scientific Management which were cool back then.

Martin is also very explicit about the small vs big team problem. In his opinion the organization of big teams is a solved problem since ages. What was open was the organization of small teams developing software. To scale from there seems to be the easy part for him. In theory, he may be right, but looking at my project which we scaled from 1 to 65 teams in years working on one product, the organism and interactions between teams and the overall organization cannot be reduced that easily. That would assume an organizational maturity on project level and on each single team that is hard to reach under realistic circumstances. I am not saying its not feasible, just that its not that simple. But Martin has shown to simplify a lot to make his points.

Another new information for me was the term „Iron Cross“ of project management:

The reason that these techniques fail so spectacularly is that the managers who use them do not understand the fundamental physics of software projects. This physics constrains all projects to obey an unassailable trade-off called the Iron Cross of project management. Good, fast, cheap, done: Pick any three you like. You can’t have the fourth.

Robert C. Martin, Clean Agile, page 15

I agree with this assessment (of course in a theoretic setup you may be able to achieve all four, but in this world this is unlikely). On the following pages, Martin describes quite typical project development in the waterfall world up until the deatch march phase.

Another example of Martin’s provocations is his statements that „The loss of hope is a major goal of Agile.“ Sounds crazy, right? What he aims at by using the metrics (like velocity) in a proper way, arbitrary and unrealistic deadlines, which are based on hopes, can be overcome, and gradually more realistic estimates can come out. This is also were he makes this statement:

Some folks think that Agile is about going fast. It’s not. It’s never been about going fast. Agile is about knowing, as early as possible, just how screwed we are.

Robert C. Martin, Clean Agile, page 27

Some pages later, Martin once again talks about the „grand redesign“ – the typical „cure“ when a software organisation has maneuvered itself into a dead end. He repeats his example from his earlier books, where a Tiger Team to redesign the software from scratch is caugth between having to catch up with the legacy software’s development while generating no income.

On page 50 Martin makes a memorable insight:

Humans make things better with time. Painters improve their paintings, songwriters improve their songs, and homeowners improve their homes. The same should be true for software. The older the software system is, the better it should be.

Robert C. Martin, Clean Agile, page 50

I really liked this statement. Software is in many cases connotated with the stance that old software/code bases are mostly getting worse, seldom better. Of course we all know good cases, mostly Open Source projects going on since decades (Linux Kernel, gcc, Latex, …), but in the proprietary software industry its often different. I cannot count how often old software really got worse over time – no matter if it was disimproved by rushed hacks or not changed at all. Martin is stating the obvious, but in that clarity I never saw that.

Many people have claimed that up-front planning is not part of Agile development. […] Of course the business needs a plan. Of course that plan must include schedule and cost. And, of course that plan should be as accurate and precise as practical. […]

In short, we cannot agree to deliver fixed scopes on hard dates. Either the scopes or the dates must be soft.

Robert C. Martin, Clean Agile, page 57f

I like this statement, because in few sentences Martin clarifies a typical debate around agile approaches. Some nitpickers – often with the aim to show the unsuitability of agile practices for tough industry business – claim that agile is anti plans or discourages plans.

Rising Velocity

If we see a positive slope, it likely does not mean that the team is actually going faster. Rather, it probably means that the project manager is putting pressure on the team to go faster. As that pressure builds, the team will unconsciously shift the value of their estimates to make it appear that they are going faster.

This is simple inflation. The points are a currency, and the team is devaluing them under external pressure. Come back to thatteam next year, and they’ll be getting millions of points done per iteration. The lesson here is that velocity is a measurement not an objective. It’s control theory 101: don’t put pressure on the thing you are measuring. […]

One way to avoid inflation is to constantly compare story estimates back to the original Golden Story, the standard against which other stories will be measured. Remember that Login was our original Golden Story, and it was estimated as 3. If a new story such as Fix Spelling Error in Menu Item has an estimate of 10, you know that some inflationary force is at work.

Robert C. Martin, Clean Agile, page 81

This is a very important observation and I can confirm it from practice. As a project manager I am monitoring team’s velocity charts. However, I am not doing this to enforce rising velocity charts, but to identify trends in underachievements (commitment vs achievements). Speaking about the average mature team, I think the best to aim for is a stable sprint velocity over longer duration of times, leading to a sustainable pace (Martin writes more on that on page 103). The last paragraph above gives a nice approach how to determine velocity inflation.

Working overtime is not a way to show your dedication to your employer. What it shows is that you are a bad planner, that you agree to deadlines to which you shouldn’t agree, that you make promises you shouldn’t make, that you are a manipulable laborer and not a professional.

This is not to say that all overtime is bad, nor that you should never work overtime. There are extenuating circumstances for which the only option is to work overtime. But they should be extremely rare. And you must be very aware that the cost of that overtime will likely be greater than the time you save on the schedule.

Robert C. Martin, Clean Agile, page 103

I really like this quote, it fits to my „natural instincts“. My work and contributions are most sustainable when I can work more or less normal hours. That means I can rest, stay sane and – more relevant to my employer – reflect on what is going on at work. As a team and project manager my day at work is usually crammed with meetings, mails and nagging people. Very rarely I have time to think deeply about what goes on. Hence, the best ideas I have after work, during weekends and vacations. I usually make sure to take notes about those ideas, so I can followup during working times.

On pages 106f Martin shares another interesting anecdote about the „developer hierarchy“ at a printer company. The baseline is, that the printer software developers enjoyed a lot of praise for their direct contributions to the company’s main business. However, that led to elitism and closing up of their code. Other software developer working in indirect matters did receive lower trust and couldnt contribute that much. I think this is something to observe also around my work. The ones directly developing fancy features which make it on magazines and news pages feel much more appreciated than the ones working in the machine room, keeping the whole machinery running. I think the boundaries between those groups must be as permissive as possible.

The point is that Continuous Integration only works if you integrate continuously.

Robert C. Martin, Clean Agile, page 108

Simple but powerful words. I cannot believe how often CI is preached and promised, while it boils down to integration of a full team’s work once a sprint or even less.

The continuous build should never break. A broken build is a Stop the Presses event. I want sirens going off. I want a big red light spinning in the CEO’s office. A broken build is a Big Effing Deal. I want all the programmers to stop what they are doing and rally around the build to get it passing again. The mantra of the team must be The Build Never Breaks.

The Cost of Cheating

There have been teams who, under the pressure of a deadline, have allowed the continuous build to remain in a failed state. This is a suicidal move. What happens is that everyone gets tired of the continuous barrage of failure emails from the continuous build server, so they remove the failing tests with the promise that they’ll go back and fix them “later.”

Robert C. Martin, Clean Agile, page 109

Not much more to say, except that this is hard in traditional environments to achieve, and very easy to loose once gained. Just recently I got the chance to focus more, which I really appreciate.

The belief that quality and discipline increase speed is a courageous belief because it will constantly be challenged by powerful but naive folks who are in a hurry.

Robert C. Martin, Clean Agile, page 134

I love that quote because it is just true. And I admit that sometimes I am one of those naive folks. However I work to be it less every day.

Towards the end of the book, Martin lays out the new movement of Software Craftmanship. I frankly didnt hear about it before, but I like the ideas about it.

As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:

Not only working software, but also well-crafted software

Not only responding to change, but also steadily adding value

Not only individuals and interactions, but also a community of professionals

Not only customer collaboration, but also productive partnerships

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.

https://manifesto.softwarecraftsmanship.org/

Many will recall the format following the Agile Manifesto. However I see it a very valuable extension from the perspective of developers.

Let me end here. Probably long enough text for anyone ever to read here 🙂

Kategorien
Coding

Mutation testing with infection

A few months ago I read about mutation testing for the first time. At that time I started my current pet project with a focus on proper testing, and the idea hooked me immediately. However, back then my overall testing state was not at a level that such advanced mechanisms would make sense. But now, during my christmas vacation, the time was right 🙂

What is mutation testing? In my own words: with mutation testing one can check the quality of unit tests for code, by automatically and intentionally mutating the code and comparing the unit test results with the original ones. The mutated code is called a „mutant“ and if your test cases recognize most of them your tests may be considered sufficient (a lot ifs and whens apply here).

For my php-based pet project, infection was the natural choice; I couldnt find any other mutation testing framework which is similarly well maintained and documented.

The initial setup, however, was not fully smooth, I had to overcome some issues in my environment:

First, I had some trouble to run my unit tests locally (so far I only executed them on my server and in CI, but I realized I can shorten my turnaround times significantly when I also enable them locally). As I – shame on me – only have Windows on my desktop, I had to setup php (with extensions such as xdebug), mariadb and Papercut (SMTP service) locally. I wanted to do this already for a while, so now I digged into it. Cost me some hours to sort everything out.

The second issue was more tricky. Some history: Due to the way how PHP and PHPUnit work (some say its even a bug in PHP?), outputting http headers via the header() function is conflicting with PHPUnit. A helpful answer on stackoverflow lists the options to solve this, and I chose the second option long time ago. The first option is not working for me, the third option didnt seem feasible either as I actually need the header output for my functionality. So I chose to run PHPUnit with the „–stderr“ parameter, worked for me. With infection however, this resulted in issues. Outputting all PHPUnit results on stderr made infection, which parses PHPUnit’s output for errors, fail. I almost got mad in resolving this „deadlock“, but then stumbled over a github issue in which some compatibility topics were discussed. As far as I understand, their issue was another one, but I tried the given workaround, which seperates the initial PHPUnit run from the mutated runs:

vendor/bin/phpunit --coverage-xml=build/logs/coverage-xml --log-junit=build/logs/junit.xml
vendor/bin/infection --skip-initial-tests --threads=$(nproc) --coverage=build/logs --show-mutations --no-interaction

This gave me the first output of infection:

The metrics at the bottom looked quite good, according to the infection documentation those values are good-ish. The hint in the second-to-last line is also important to acknowledge: Not all slipped-through mutants are in fact problems. Anyhow, it also shows 26 not covered mutants, i.e. code mutations which were not uncovered by my existing test cases.

In the infection.log file you can see which of those there are. Example (redacted):

2) ...\TaskController.php:137    [M] NewObject

--- Original
+++ New
@@ @@
         if (empty($taskId)) {
-            return new Response('HTTP/1.1 500 Internal Server Error ');
+            new Response('HTTP/1.1 500 Internal Server Error ');
+            return null;
         }     

Seems my test cases are not checking the return-types sufficiently or my code is not further using the returned objects here. Interesting! As a next step I will investigate how I can improve my unit tests based on the mutation test results.

Kategorien
Book

Thougts on Clean Architecture from Robert C. Martin

After recently reading Clean Code from the same auther, now I followup on Clean Architecture from Robert C. Martin. While Clean Code focusses mostly on the smallest entities of software development and in a simplified manner can be called bottom-up, Clean Architecture naturally comes from top-down. Clean Code is much easier to apply in examples such as my current pet project, Clean Architecture is something more abstract for me, as I do not really have a huge software project at hand to apply.

Down the darkest path comes the idea that strong and stable architecture comes from authority and rigidity. The architect’s mandate is total and totalitarian, the architecture becoming a dystopia for its developers and a constant source of frustration for all.

Down another path comes a strong smell of speculative generality. A route filled with hardcoded guesswork, countless parameters, tombs of dead code, and more accidental complexity than you can shake a maintenance budget at.

The path we are most interested is the cleanest one. It plays more to our strengths than our weaknesses. We create things and we discover things. We ask questions and we run experiments. A good architecture comes from understanding it more as a journey than a destination, more an ongoing process of enquiry than a frozen artefact.

Kevlin Henney in the Foreword

Kevlin’s words chime very well with my personal understanding of good software architect mindset. In my career, I have seen all of the above, and I am glad in my current project all architects have the latter. However, its not a closed book. Many managers, suppliers and developers still demand for what I would call static enterprise architecture, with different rationale behind. While management sees architecture as a definition of tools, libraries and complete set of boxes to ultimately save money by „clarity“, suppliers want something fixed to make an offer towards and have something to blame when problems arise.

I’ve seen it happen. I’veworked in projects where the design and architecture of the system made it easy to writeand easy to maintain. I’ve experienced projects that required a fraction of theanticipated human resources. I’ve worked on systems that had extremely low defectrates. I’ve seen the extraordinary effect that good software architecture can have on asystem, a project, and a team. I’ve been to the promised land.

Introduction

This is an important statement, easy to overlook and ignore. Most of us know reality well enough to have skepticism regarding the applicability of clean approaches to corporate software development. The constant stress by deadlines, tight budget, competing stakeholder’s requirements and technology fights often overrules the best intentions. But giving up is no option. Every new project, every new week its worth to keep trying to establish a bit better code, a bit better architecture. Our own future selfs, and our professional spirits will be thankful.

The goal of software architecture is to minimize the human resources required to build and maintain the required system.

What is design and architecture?

I am not sure if I would call the above „The“ goal, but certainly its „a“ central goal. Replacing „human resourcues“ by „effort“ for me would make this goal statement more generally applicable. Effort can include human resources, but also license cost, mental imbalance and turnover.

On pages 5ff, Martin gives a case study of a company sinking exponentially more engineering staff while creating exponentially less output. Interesting and highly relatable. I hope I find ways to calculate according metrics also for my private and professional projects.

And that’s the answer to the executive’s dilemma. The only way to reverse the decline in productivity and the increase in cost is to get the developers to stop thinking like the overconfident Hare and start taking responsibility for the mess that they’ve made.

The developers may think that the answer is to start over from scratch and redesign the whole system—but that’s just the Hare talking again. The same overconfidence that led to the mess is now telling them that they can build it better if only they can start the race over. The reality is less rosy:

Their overconfidence will drive the redesign into the same mess as the original project.

What is design and architecture?

Martin is focussing here on the software developers and provoking them to not take management influence as an excuse for their own failure to maintain sane software projects. He has done the same in Clean Code. Personally, I can see both sides. Of course management often overrules developers to gain what they call „speed“ by effectively telling them to hack some dirty solution together. On the other hand I have seen enough developers either having a messy approach or giving up on management on the slightest push. So being somewhat in between I always try to balance and find tradeoffs. It sounds tempting to target only clean approaches and refuse deadlines completely, but its unrealistic and being too hard here my backfire. I know a developer who is so strict in his believes (and judging only by the technical content he is always right), that he is notorious to be ignored. Who wants to work with somebody who is not willing to compromise at all? (Me, yes, but many other managers take the short path and just go to the next developer)

Thus the final version of the SRP is:

A module should be responsible to one, and only one, actor.

SRP: The Single Responsibility Principle

Finally I got it. For a long time I really struggled to internalize the Single Responsibility Principle. All the other definitions or descriptions kindof made sense, but failed me to be applicable in a crisp way. With the above definition using the actor I got it 🙂 Apart from the SRP, Martin of course also introduces also the other SOLID principles.

While the SOLID principles apply to class interdependencies, he continues to establish three principles on component interdependencies. Naturally, they relate to the SOLID principles, so I wondered if there are ways to generalize a set of principles to apply them on any level of software, from systems via components to classes (and maybe even functions).

The Reuse/Release Equivalence Principle:

The granule of reuse is the granule of release

Component Cohesion

I really liked this principle, it sounds so obvious and clear, however I never thought about release cadence as architectural principle. Nice!

The Common Closure Principle and the Common Reuse Principle are very similar to the Single Responsibility Principle (see above) and the Interface Segregation Principle from SOLID, so both didn’t sound so new.

In the section on Component Coupling, Martin introduces two interesting metrics: Instability and Abstraction. After some reasoning, he defines the area around low instability and abstraction as the zone of pain and the area around high instability and abstraction as zone of uselessness. Classes on the „main sequence“ in the diagonal from (0/1) to (1/0) instead are considered sane. Exceptions of course apply, e.g. for database schemes (stable and very concrete).

I checked around and really could find a tool which calculates and displays the metrics for my pet project code. I directly added it to my CI pipelines. See below the recent output. Seems I need to check some of my code.

The strategy […] is to leave as many options open as possible, for as long as possible.

What is architecture?

Only 135 pages in Martin asks the question „what is architecture?“, and his answers really surprised me, it was nothing I really expected. In the following pages he gives many examples (really many, probably too many) for this. For Martin, „details“ such as external interfaces (IO, databases), frameworks, the web are not relevant for the core business rules. The latter are for him the core and to be untainted from all such details.

A good architect maximizes the number of decisions not made.

What is architecture?

I think again this is the provocative author, but in the context of his definition of „decision“ and „details“ this makes sense.

And then, thankfully, the Agile Software Development revolution arrived and put architects like me out of our misery. I’m a programmer. I like programming. And the best way I’ve found to have a positive impact on code is to write it.

The dinosaurs of Big Architecture—typically to be found wandering the primeval plains of Big Process—were wiped out by the asteroid of Extreme Programming. And it came as a blessed relief.

[…] The problem with leaving architecture to programmers is that programmers have to be able to think like architects. It turns out that not all of the stuff we learned during the Big Architecture era was of no value. The way that software is structured can have a profound impact on our ability to keep adapting and evolving it, even in the short term.

[…] Learn how you can incorporate design principles and Clean Architecture into your development processes […]. Perhaps, before you commit code, ask a colleague to review it with you. And look into the possibility of adding a code “quality gate” to your build pipeline as a last line of defense against unclean architecture. (And if you don’t have a build pipeline, maybe it’s time to create one?)

Most important of all is to talk about Clean Architecture. Talk about it with your team. Talk about it with the wider developer community. Quality is everybody’s business, and it’s important to reach a consensus about the difference between good and bad architecture.

Be mindful that most software developers are not very architecture-aware, just as I wasn’t 25 years ago. More experienced developers clued me into it. Once you’ve wrapped your head around Clean Architecture, take the time to wrap someone else’s head around it. Pay it forward.

[…] The real journey starts here.

Jason Gorman in the Afterword

I think thats perfect last words without any more need of commenting!

Kategorien
Coding

State of Pipelines

Just a brief update on my current pet project’s Jenkins pipelines setup. The Jenkins BlueOcean graph probably already covers it mostly:

I have parallelized some „Fast Tests“ – all of these take less than a second to run currently due to the small codebase. If any of them fails it doesnt make sense to proceed. Currently these contain

  • phpmd: „mess detector“ using the clean code ruleset to identify unclean code.
  • pdepend: Some code analytics I learned while reading „Clean Architecture“ (review will come to the blog soon I hope).
  • phan: Static Code Analysis
  • phpunit: Unit testing

After those I run API tests with schemathesis. This takes one minute (configurable) and sends random payload to my REST API as per the generated swagger file.

Last but not least I run some acceptance tests, as described in Adventures with PHPUnit, geckodriver and selenium.

For a while I also parallelized the latter two test activities in a „Slow Tests“ block, but I learned that the test execution was very unreliable due to the load the test environments put on my server. So for now they run sequential.

Today I finished some major clean code refactoring and I am happy to say that I could rework most of the code and the tests in place ensured that functionally nothing broke without me checking the actual application manually once! This is a major achievement for me personally, as all my former pet projects based on heavy manual testing on basically every change.

Kategorien
Coding

Adventures with PHPUnit, geckodriver and selenium

In my current pet project, I try to not only focus on the features itself, but also in the quality of the software and the development environment. Since I programmed code myself, the professional software development world has evolved quite a lot, and new approaches and tools have emerged and are de facto standards nowadays. While I know them as an observer, I barely used them myself. That I want to change.

Historically, my project’s setup already included a selenium test file, using geckodriver and Facebook’s php-webdriver library. However, the test code itself was a plain php file, executing a series of webdriver instructions and returning failures when some string comparisons did not yield the desired results:

<?php

namespace Facebook\WebDriver;

use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;

$host = 'http://localhost:4444';

$driver = RemoteWebDriver::create($host);

$driver->get('https://example.com/foo.php');

$driver->findElement(WebDriverBy::id('title'))
       ->sendKeys('Acceptance test title'); 

...

if (strcmp($driver->findElement(WebDriverBy::id('title'))->getAttribute('value'), 'Acceptance test title')!=0) {
    echo "Just created task cannot be accessed!";
    $driver->quit();
    exit(1);
}

$driver->quit();

This code worked, and helped me to assure that my product didn’t break (to some degree) from a user perspective. However, it was ugly to maintain, didn’t produce any helpful test report and just didn’t feel right.

Failed Approach

After some research I found out about phpunit-selenium, a project combining the powers of phpunit, I guess the dominating unit testing framework in the php space with selenium. Interesting. However, from there I went down a slippery road to trial-and-error-land. I should have probably get cautious, when the only documentation I found was referring to phpunit 3.7 (its now at 9.2), and even the github project was only referring to phpunit 8.x versions. With some back-and-forth, I could identify a version which fitted together with phpunit in composer repos and sucessfully deployed:

{
    "require-dev": {
        "phpunit/phpunit": "9.2",
        "phpunit/phpunit-selenium": "dev-master"
    },
}

Apart from that, I had to install the selenium server, as so far I was directly connecting to the geckodriver. This led to the worst part of issues: For some reasons in 95% of cases, the test cases didnt execute properly, and it was somehow related to firefox crashes. I tried so many things:

  • Made sure both firefox and geckodriver had most recent versions (they had), same for selenium server.
  • Executed everything as non-root.
  • Tweaked permissions.
  • Increased log-levels of geckodriver, selenium server and even firefox (the latter didnt even create crash dumps).
  • I even started to investigate if my server’s ubuntu has some limiting settings in systemd cgroups, somehow limiting the number of parallel threads and memory. After all, its a browser we are launching here.

All of the above didnt really change anything. It was really frustrating, and the whole chain was quite long to debug in my spare time. After approx. 10 hours of frustrated tests, I gave up and made up my mind.

Solution

Luckily, I found another approach, based on the same tooling I had initially already working, but in a cleaner way. How does this look like? I simply combined phpunit and webdriver calls without any third party addition. Code says more than thousand words:

<?php declare(strict_types=1);

namespace Facebook\WebDriver;

use PHPUnit\Framework\TestCase;
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;
use Facebook\WebDriver\Exception\UnknownErrorException;

final class AcceptanceTest extends TestCase
{
    protected static $driver;

    public static function setUpBeforeClass(): void 
    {
        $host = 'http://localhost:4445';

        $capabilities = DesiredCapabilities::firefox();
        $capabilities->setCapability('moz:firefoxOptions', ['args' => ['-headless']]);
        $capabilities->setPlatform(WebDriverPlatform::LINUX);

        self::$driver = RemoteWebDriver::create($host, $capabilities);
    }
    
    public static function tearDownAfterClass(): void
    {
        self::$driver->quit();
    }

    public function testTaskIsSuccessfullyCreatedFromContent()
    {
        self::$driver->findElement(WebDriverBy::id('title')) 
             ->sendKeys('Acceptance test title'); 

...

        $this->assertEquals('Acceptance test title', self::$driver->findElement(WebDriverBy::id('title'))->getAttribute('value'));
    }
}

This gives me the full power of phpunit, with fixtures, asserts, reports and a nice console output:

PHPUnit 9.2.0 by Sebastian Bergmann and contributors.

..                                             2 / 2 (100%)

Time: 00:14.702, Memory: 6.00 MB

OK (2 tests, 3 assertions)
Kategorien
Book

Thoughts on „Clean Code“ from Robert C. Martin

I recently ordered some books from my project’s book recommendation list. This list is maintained by a few colleagues and contains many classics, but also some more hidden gems. Clean Code probably had to be on that list, and its not my first encounter. I skipped through the book already a few occasions before, but never really had the time and motivation to really digest its content. This time, this was different.

This is not a book review, but rather some personal notes and thoughts I gathered while reading it.

I particularly liked the first chapter, because it gathers quite some good statements and arguments why clean code really matters. Of course, everyone who ever development reasonably sized software themselves, know about the pros of clean code (as per whatever definition/guideline/philosophy). However, in daily close combat with management, customers and fellow developers, its not always easy to have the perfect argumentation at hand. So I always like to learn about new arguments objectively supporting subjective opinions.

The section „The Grand Redesign in the Sky“ describes very picturesque who the „its such a mess, lets do it from scratch“ often tends to (not) pan out in reality. That fits nicely to earlier thoughts and reads of mine regarding featuritis and the competition of legacy code maintainers and the new tiger team starting from scratch. Martin’s anecdote about such a competation taking more than 10 years was really eye-opening.

The next section talks about the attitude of developers needed to defend clean code:

Attitude

[…] Why does good code rot so quickly into bad code? We have lots of explanations for it. We complain that the requirements changed in ways that thwart the original design. We bemoan the schedules that were too tight to do things right. We blather about stupid managers and intolerant customers and useless marketing types and telephone sanitizers. But the fault, dear Dilbert, is not in our stars, but in ourselves. We are unprofessional.

This may be a bitter pill to swallow. How could this mess be our fault? What about the requirements? What about the schedule? What about the stupid managers and the useless marketing types? Don’t they bear some of the blame?

No. The managers and marketers look to us for the information they need to make promises and commitments; and even when they don’t look to us, we should not be shy about telling them what we think. The users look to us to validate the way the requirements will fit into the system. The project managers look to us to help work out the schedule. We are deeply complicit in the planning of the project and share a great deal of the responsibility for any failures; especially if those failures have to do with bad code!

Robert C. Martin: Clean Code, Pearson Education, 2009

I can very much relate to that code and – as a engineering manager and project manager with developer history – support confident developers defending their ground. Of course, there are more shades of grey, and I have experienced many arguments with developers who went too far in unconcrete demands for more time because of better quality. In my perspective, a proper discussion based on facts, figures and clear arguments from both sides – management and developers – yields the most best results, preferably as a compromise. But I get why Martin is triggering developers here explicitly.

Following are some statements from famous developers, you can read the first chapter of the book for free here. Then the actual content begins, where Martin describes one technique/rule/guideline after another, better naming, function design, commenting, formatting, error handling, testing etc. What follows are some refactorings of different sized open source code snippets, piece by piece, and in the end a list of „smells and heuristics“, an enumeration of indicators for bad code.

I have tried to apply the knowledge I gathered from reading this book in two ways:

  1. I added phpmd – php mess detection – to my current pet project, with the clean code rules. Of course it can only identify a subset of bad code indicators, but its already a good start. There were some findings which I could quickly fix.
  2. More practically, I started to refactor the code. E.g. I removed duplicate code, reduced function sizes and improved the error handling. Some of those improvements I already merged on master, but I am definetly not yet finished.

The worst part of the book was the example for „improved“ code given on pages 50ff. Maybe I didn’t get something essential, but this code is super-hard to read, understand, and totally cluttered with 1 to 3 line functions with awkward naming. I even suspected that this was kind of a trap/test to trigger the reader’s objection. Seems not. This made me also google for critique on the book, and this blog article also puts some good stances on it.

Can I recommend the book? Yes. I would take basically everything in it with a grain of salt, and there are some elements which I find „too much“. With that in mind, its a good motivation to critically reflect on own code and improve it iteratively.

Kategorien
Culture

How some distrust in modern software development is good

A few weeks ago a colleage mentioned in an argument that in contrast to open source communities, where mutual distrust between many personally unknown contributors is prevalent, in a corporate environment due to the contracts of all developers trust can be assumed. Therefore, some specific – but out-of-scope of this article – practices cannot be taken over to corporate environments. It made me think.

I concluded today that there has to be a differentiation. One has to distinct between trust in people and trust in code/results.

It may be true or not true, that in corporate environments there is more trust, because many contributors know each other personally from regular meetings, sitting next to each other in an office and, maybe most influencial, by the nature of their contracts. Of course we all are also aware of the sources of corporate distrust, mainly politics and personal agendas. One could also argue that in Open Source projects there is more trust among contributors because of the mutual mindset, nice conferences and legends of old mailing list battles against each other or united against some guys from the other paradigm (indentation anyone?). I would say its in every organization’s highest interest, to nurture and foster trust among the developers, most importantly trust in the fact that all are doing the best they can with best intentions. An organization in which people think others are intentionally blocking or even destroying their progress is toxic and will lead to significant problems sooner or later (or needs to be covered with effort, money, time, …).

My point is that trust in code is never good. Its a bold statement, but look at your own environment. Are you 100% trusting the code you work with? I would not. Do you trust the Pull Request from someone without looking at it? I would not. Do you trust your own code you have produced today? Yeah maybe, but when you look at it in one year you should have at least a critical perspective. Some genius developers think their code is always perfect (and it may very often be). I dont think it is. There is also strong evidence that thinking your code is perfect doesnt make you a good developer. A good developer knows about the dangers coming with every written line of code. Almost every code can be source for trouble. It can contain bugs, it may call other code which is buggy. Even if its absolutely bug-free today, tomorrow a compiler issue, weird build pipeline or hardware glitches may break it (and rather than solving the root cause you may be forced to update your code with a nasty workaround). Some may say that viewpoint is depressing, and it is a bit. But neglecting the facts is not sustainable. It is the daily struggle of our industry.

Coming back to my colleague’s argument I would say: In our corporate project we have about the same amount of trust among team members like in an open source community of comparible size and we work hard every day to improve the basis for trustful collaboration. But we should also be always distrustful with the code we produce ourselves and the code we get from external parties.

And there is and should be no difference between open source projects and corporate environments regarding trust in people and distrust in code.

Looking forward to get your comments on this.

(I posted this article also on LinkedIn)

Kategorien
Archive

Master thesis presentation

I found again my presentation on my master thesis from 2011. It was about „Prüfung intervallbasierter automobiler Konfigurationsdaten“. This topic combined my interest for logic, theoretical computer science with a real application in the automotive industry. Check it out!

Master thesis presentationHerunterladen