(Fact 8.) One of the two most common causes of runaway projects is poor estimation.
(Fact 9.) Software estimation usually occurs at the wrong time. - at the beginning of project.
(Fact 11.) Software estimates are rarely corrected as the project proceeds.
Consequence: Re-estimate your issues!
This an an extremely powerful technique to control estimation and schedule plan.
Take your time (at 1/3 of the estimates project schedule) and review all task/story remaining and adjust estimates based on experiences. At that time you have gained a solid experience in domain, technology so your estimate will be much more reliable.
What is more: it is absolutely independent from any other estimation technique. It can be applied on any kind of estimates: ideal man day, story points, confidential ranges etc.
You might be afraid of spending huge amount of time on estimating again. But it is wrong. The first estimation takes long but if you analyze how you are spending your time you must recognize a certain pattern: it takes more to understand the issue then doing the estimate.
But the second time you do not need to understand the domain. You can focus on estimation only.
Based on my experience it takes fraction of time comparing to the time spent on the first estimate.
Several months of work can be reviewed/re-estimated in 1 hour with team. It means it can be really frequent. My experience is that after the first or second re-estimation no more re-estimation is needed because the domain become clear the estimations are not reducing risk significantly.
I have ambivalent feeling related to this book.
First time I have read I thought that this book is obvious. (At that time I was reading many of the book it is referring to).
The second occasion I have realized that from an objective point of view it is a great book.
Reading the third time (after partially reading "How to measure anything") I had to admit how important the book is. It is full of evidences, statistical facts, studies to prove how valuable each of the statement it has.
The most important factor in software work is the quality of the programmers.
The best programmers are up to 28 times better than the worst programmers.
Adding people to a late project makes it later.
The working environment has a profound impact on productivity and quality.
Tools and Techniques
Hype (about tools and technology) is a plague on the house of software.
New tools and techniques cause an initial loss of productivity / quality.
Software developers talk a lot about tools, but seldom use them.
One of the two most common causes of runaway projects is poor estimation.
Software estimation usually occurs at the wrong time.
Software estimation is usually done by the wrong people.
Software estimates are rarely corrected as the project proceeds.
It is not surprising that software estimates are bad. But we live and die by them anyway!
There is a disconnect between software management and their programmers.
The answer to a feasibility study is almost always “yes”.
Reuse-in-the-small is a solved problem.
Reuse-in-the-large remains a mostly unsolved problem.
Reuse-in-the-large works best in families of related systems.
Reusable components are three times as hard to build and should be tried out in three different settings.
Modification of reused code is particularly error-prone.
Design pattern reuse is one solution to the problems of code reuse.
For every 25 percent increase in problem complexity, there is a 100 percent increase in solution complexity.
Eighty percent of software work is intellectual. A fair amount of it is creative. Little of it is clerical.
One of the two most common causes of runaway projects is unstable requirements.
Requirements errors are the most expensive to fix during production.
Missing requirements are the hardest requirements errors to correct.
Explicit requirements ‘explode’ as implicit requirements for a solution evolve.
There is seldom one best design solution to a software problem.
Design is a complex, iterative process. Initial design solutions are usually wrong and certainly not optimal.
Designer ‘primitives’ rarely match programmer ‘primitives’.
COBOL is a very bad language, but all the others are so much worse.
Error removal is the most time-consuming phase of the lifecycle.
Software is usually tested at best to the 55 to 60 percent coverage level.
One hundred percent test coverage is still far from enough.
Test tools are essential, but rarely used.
Test automation rarely is. Most testing activities cannot be automated.
Programmer-created, built-in debug code is an important supplement to testing tools.
Reviews and Inspections
Rigorous inspections can remove up to 90 percent of errors before the first test case is run.
Rigorous inspections should not replace testing.
Post-delivery reviews, postmortems, and retrospectives are important and seldom performed.
Reviews are both technical and sociological, and both factors must be accommodated.
Maintenance typically consumes 40 to 80 percent of software costs. It is probably the most important software lifecycle phase.
Enhancements represent roughly 60 percent of maintenance costs.
Maintenance is a solution– not a problem.
Understanding the existing product is the most difficult maintenance task.
Better methods lead to more maintenance, not less.
Quality is a collection of attributes.
Quality is not user satisfaction, meeting requirements, achieving cost and schedule, or reliability.
There are errors that most programmers tend to make.
Errors tend to cluster.
There is no single best approach to software error removal.
Residual errors will always persist. The goal should be to minimize or eliminate severe errors.
Efficiency stems more from good design than good coding.
High-order language code can be about 90 percent as efficient as comparable assembler code.
There are tradeoffs between optimizing for time and optimizing for space.
Many researchers advocate rather than investigate.
And the list of fallacies:
You can’t manage what you can’t measure.
You can manage quality into a software product.
Programming can and should be egoless.
Tools and Techniques
Tools and techniques: one size fits all.
Software needs more methodologies.
To estimate cost and schedule, first estimate lines of code.
Random test input is a good way to optimize testing.
“Given enough eyeballs, all bugs are shallow”.
The way to predict future maintenance costs and to make product replacement decisions is to look at past cost data.
You teach people how to program by showing them how to write programs.
You can see story point more and more famous as a kind of estimation (and scheduling technique). It intends to simplify estimation and scheduling but in practice it makes life more and more difficult.
What is the story point?
Let’s see some definitions:
“Story point is a arbitrary measure used by Scrum teams. This is used to measure the effort required to implement a story. In simple terms its a number that tells the team how hard the story is. Hard could be related to complexity, Unknowns and effort.” – agilefaq
“A story point is to program code what a kilogram is to sand or a kilometer is to distance: An arbitrary unit of measure which describes how heavy, far, big or complex something is.” – Explaining Story Points to Management (Otto: Soooo bad…., see later, mixing up SP and velocity and their relationship…)
” The number of use case points in a project is a function of the following:
- the number and complexity of the use cases in the system
- the number and complexity of the actors on the system
- various non-functional requirements (such as portability, performance, maintainability) that are not written as use cases
- the environment in which the project will be developed (such as the language, the team’s motivation, and so on) “ – Estimating With Use Case Points (Otto: Quite a lot of not so related aspect of development…)
And many many more….
It has a huge literature. One of the always quoted reference is Mike Cohn’s Agile estimating and planning. It is not only about story point but it is clear that Cohn preferes story point more. (On the other hand the book itself is great, a must read.) (De facto “main reference point” about story points.)
In out “everyday” it come up in the context of Scrum I have participated in several Scrum introduction and I have indirect (but close enough to be reliable) information from other. Without any exception it introduced the usage of story points.
BUT: Scrum is not about story points. Not at all. What is more! Not even talking about estimation. Nothing. When I am saying Scrum I am talking about the Scrum as it is. Of course when you are talking about Scrum it is always the original Scrum PLUS many-many additional tools and techniques (including story points).
Once I had a talk with some manager using Scrum in his organization. He told me that one of their current challenge (using Scrum for many years and still having this problem. hmmm…. - on the other hand they have introduced a very agile process which is great) is to make business people understanding the concept of story points. I think it is a mistake to explain it to a businessman who is interested in time and schedule…
In one of his presentation Dan North highlighted why story point is insane (around 24:40 - not exact transcript):
- (businessman) How much will it cost and when can I have it?
- (agile guy) We don’t know. We are agile.
- What?… How much will it cost and when can I have it? These are not hard questions.
- OK. We have done some work and we think that it gonna be 295 stories..
- And about 1000 story points.
- What is the story point?
- We don’t know yet.
- You are absolutely kidding me.
- We gonna run for few weeks and we gonna burn up and burn down and the velocity and hablalal…
- Stop! Stop right now and get out of my building.
(Humorous with lots of truth!)
As I see (and it cause me many problem when dealing with story points). It tries to mix up many unrelated estimation characteristics into one single number:
- complexity: more complex job, challenge of new technologies are higher in story points. But later the same level of story become less because of gained experience. Although the same kind of taks should have the same story point.
- work volume: more work should be bigger, obvious.
- team experience: less experienced team in certain area is giving more story point then experienced team. (as story point is sensitive to team composition it becomes clear by time: In the beginning the same task has higher story point then later)
- individual developer experience: better developer can be more than 10X more productive and faster then a dump. And as all team is composed by different qualities of developer…. A single number is hiding capabilities and risk of different developer qualities.
As these risks are so independent from each other you have to use different strategies and techniques to manage them. But if you are hiding these aspect behind a single figure you do not even have a chance.
Plus it has so many other weakness:
- any kind of story point estimate (and its brother velocity ) is unique to the team. Uncomparable. How to deal with multiple team project portfolio? It has to be transformed to a common measurement and measurement unit (most probably calendar days)
- extremely sensitive to team composition change. Someone got sick? A developer has an urgent support work in other project? Or just got a new team member? Or someone quits? (statistically it is quite probable in any project takes more than half a year)
- not giving any information without velocity . Obvious. After having velocity you could estimate due dates (which is a calendar unit!!!!!) and costs (which is calendar unit and money!!!!).
- there is no strong correlation amongst story point and real effort spent. The only thing can be sure that higher story point is more then lower. But a 4 sp is not 2 times more then 2 sp! (disclaimer: correlation exists but not strong except on small stories and small story points - but it leads to #noestimate subject)
So if story points has so many issues why are we using it?
Instead of story points
Instead of story points you should use calendar based estimates with proper risk management and (semi-)automatic estimation adjustment.
But how? It is a subject of another article (later).
Original: 2009 on http://takacsot.blog.hu
– en –
Must read book.
I had two sources which made me interested in the book. The first is a presentation in Youtube. The second is a book about Scrum. Both of sources was persuasive enough to read the book. So I did. An I was not disappointed.
There are lots of subject and experience described which is good not to expereince yourself. You could nmake a much faster Scrum introduction if you know what is described in the book.
– hu –
Két forrás keltette fel a kiváncsiságomat ez iránt a könyv iránt. Az egyik egy előadás, amit a Youtube-on láttam. És egy a Scrum-ról szóló könyvel kapcsolatban. Mind a két forrás meggyőző volt. Így hát neki estem. És nem csalódtam. Kiválló könyv. Sok olyan témával és tapasztalattal, amit jobb, ha az ember nem saját maga tapasztal ki, ha nem muszály. Sokkal hamarabb lehet bevezetni az agile fejlesztési módzsereket, ha tisztában vagyunk a könyvben leírtakkal.
Teljesen biztos vagyok abban, hogy még párszor el fogom olvasni az idén.
Why I hate hibernate
It is series of articles highlighting difficult or even impossible to resolve issues with Hibernate (in Grails). And his is right.
I summarize my experience like this:
For simple things Hibernate is very good but all other mapping tools are good too. Bus as soon as you reach a certain level you have to lick Hibernate’s ass. It is not serving you anymore but you are serving it.
- I don’t like Hibernate (and Grails), PART 1
- I don’t like Hibernate/Grails, part 2, repeatable finder problem: trust in nothing!
- I don’t like Grails/Hibernate part 3. DuplicateKeyException: Catch it if you can.
- I don’t like Grails/Hibernate, part 4. Hibernate proxy objects.
- I don’t like Hibernate/Grails part 5: auto-saving and auto-flushing
- I don’t like Hibernate/Grails part 6, how to save objects using refresh()
- I don’t like Hibernate/Grails part 7: working on more complex project
- I don’t like Hibernate/Grails, part 8, but some like Hibernate and Grails. Why?
- I don’t like Hibernate/Grails part 9: Testable code
- I don’t like Hibernate/Grails part 10: Repeatable finder, lessons learned
- I don’t like Hibernate/Grails part 11. Final thoughts.
there are three factors for work motivation:
- Autonomy: the control you have over your work. The more you can control what to do and when to do it, the more motivated you are.
- Mastery: the sense of progress you get. The more you think you are getting better at what you do, the more motivated you are.
- Purpose: the meaning you get from your work. The more what you do matters to you, the more motivated you are.
once you reach the middle class, you should focus on how you make money rather than how much. The way becomes more important than the amount.
Autonomy could be difficult to achieve if you work for someone else…. increase your bargaining power by being very good at what you do. Your company will be willing to give you more freedom if it really needs you. Another way is to build your own business.
Do you remember when the word “responsive” meant fast?
But now, all of a sudden, the word means: compatible with mobile devices.
what the word “respond” means. You respond to a stimulus. A screen is not a stimulus. A screen is an IO device. You don’t respond to the format of an IO device. You respond to Users. Consarned kids.
Craftsmen are arrogant.
craftsman in practice—it needs to be done perfectly and proudly. Their focus is fully on minor details, because those create the base from which their skill level can grow.
Software skill exists on an asymptotic curve. When you first learn something, you make large strides immediately and feel like you’re making serious progress toward perfection. But as time goes on, and as you increase your familiarity with the subject, your progress becomes more incremental.
They challenge others to climb the walls and jump to try to close this gap too, and they employ hyperbole to justify the importance of seemingly mundane details and practices. “We need to test this remote edge case.” “Why haven’t you refactored this class?” Craftsmen are constantly working toward an ideal, and any ideal other than perfection is offensive. This hyperbole is not necessary, but it comes from a bias.
This bias creates a communication issue. Craftsmen are so narrowly focused on closing the gap between themselves and mastery that it becomes difficult to consider that others might not share this focus.
They sound arrogant because their language is drawn from a different set of assumptions.
Craftsmen are slow.
In a fast-paced, results-oriented team, a craftsman spends time refactoring working software. This can seem academic or self-indulgent in the face of a deadline. But the reason is both simple and practical.
As craftsmen, we’ve done enough practice to know that the cost of work has a non-linear relationship to its quality. If you don’t take advantage of an opportunity to refactor and increase the quality of your code early, it will take exponentially longer to refactor and add features later. Your code will have more bugs, and will be difficult to extend when your clients need added functionality the most.
Craftsmen are dogmatic.
The first time you neglect to write a test, you increase the likelihood that you’ll neglect to write a test in the future. You’ll develop poor habits, and your skills will slowly deteriorate.
Peter Smith - It doesn’t work like that in enterprise
Definition of Enterprise development
- Constrained process and practices (sometime the lack of it)
- As long as a sub-optimal processes profitable, it will be allowed to continue (Otto: so you(!) have to prove that you change is more profitable)
- Take Control
- Don’t stop being lay developer (automate)
- If you tru had enough there will be ways in which you can speed yourself up
- Consider committing your own time (!!!!!)
- Keep innovating, and do not give up.
- Planning for the future
- Most enterprise is planning for the future to some extend
- Fire-fighting is much more common than working proactively
- Sometime there is no management of technical dept at all
- it can be very difficult to communicate the true value of writing/refactoring code to be loose coupled and maintainable
- Take control
- Start the conversation! - Yes it is politics…
- Lead by Example
- ask your architect or development manager what the
self-lifeof your software is (Otto - do not really get this advice ????)
- Rejection of industry standard knowledge
- Companies tend to shy away from buy to things they perceive as teny
- Individuals often try to dins the cleverest solution and ignore the simple one right in from the them,
- Most teams should be able to react quickly to client demands regardless of their processes of failing in other areas.
- Take control
- Share your knowledge
- Demonstrate (cool) useful things
- Know the why as well as the what and the how
- Help people to find the art and the joy in coding
- Management and people
- Developers often treated as liability rather than a resource
- Staff are often “promoted” based on length of service rather than proficiency and suitability for the role
- People with responsibility tend to subconsciously protect their position and their interest
- Take control
- Remain true to yourself and uphold your own standard (be proud of it)
- Say no
- The golden rule: “ Am I still learning skills that will be relevant to me in my future career?”
What is the solution?
Scott Allen - Building Directives for AngularJS
Meg kell tanulnom AngularJS-t Mese nincs.
Demonstrated how to create custom tags into HTML and have some behavior around it.
Craft Conf presentations
More like a motivational speech but very good. You want to be a master of development.
Until you are novice pretend to be master
But being a master you should be able to use any practice: TDD, SOLID with critique…
Practice until you realize when it is useful.
As a journeyman
- try different approaches
- try different domains
Learn how to learn
Listen like you do not know the answer - you may not know.
Amazingly good presentation.
It is focusing how to make agile really working and that is not really about developers but they are wrong too…. I have to re-watch to extract.
We have to change how we are thinking about user stories.
User story is about behavior changes.
How software sucks
Specification are extremely difficult to get right.
- human language is not the best for that
- humans are sux
- stakeholders are not the real stakeholders
laziness is promoted as something good.
- brainless way. People use tool which is not planned for that purpose because they are lazy to find/develop the proper tool.
Learning curve of huge projects is amazingly huge. It become difficult to add features quickly. (!of course not decoupled!)
Learn to hit the fucking deadline:
- Learn estimating
Learn papers: there are many researches about how to develop big system and resolve problems effectively (heheh történt ilyen mikor valaki bemutat egy design patternt mint valami újdonság és jó design cuccost)
Engineer are function better autonomously
- decide on what and how to work and on what
You are judged based on your API
So: choose the proper size of component to be able to work autonomously using well defined API to communicate each other. So build component maintainable.
DO not refactor but rewrite component. Rewrite better, based on knowledge you know about. It is part of maintenance.
SQL is not API!!!
War stories with humour and fun. Very interesting.
- 20 architect -> It is sure it will not work :)
- Don’t cache the cache.
- In distributed systems, application state is your enemy
- If it is “sophisticated”, you probably shouldn’t be doing it
- Development != production
- Data should be free from code dependency. Data is just data. Especially not for specific version of applications.
- The longer you store the simpler data should be.
- Beware of smelly BLOBs
- Your system WILL be dynamic
- Centralized responsibility hurts. Single person bottleneck
- Face too much rigidity, a way around the rules will be found
- If it makes you want to scratch your eyes out, you might not want to do it.
- Sometimes constraints need to be fought
- Beware all things “meta”
- Fight the madness
- Tunneling is the root of all evil
- Don’t be too smart
- Do not try to build system which fits to everyone
- Don’t be all things to all people
- Sometimes specific code is preferable
- Saying no might help sometimes
- Give feedback
Individual goal are against company goals because most of th goal need teamwork which is killed by individual goal setups.
and many other mut it was not so structured to extract the “single message”.
Use values instead of estimates (business values….) for deciding to initiate the project. There are many way to define value (ROI, Risks, Cost of delay, Importance, Willingness for investment)
Plan in different levels: roadmap, long term, mid term, short term
All videos can be found on http://vimeo.com/ndcoslo/videos
Doc Norton - The Technical Debt Trap
The danger occurs when the dept is not paid back. Every minute spent on not so good code counts as interest in that dept. Technical dept is good.
- rapid delivery, time to market
- heps learning, indication of learning
People are confused the dept metaphor with the idea that you could write code poorly.
Code must be clean enough to be able to refactor! (always) - Otto: so Tests are there…., well tested :)
- is the code clean?
- is the code tested
- is there a learning objective or event
- is there a plan for payback
- is the business truly informed
I any of these are missing it is not technical dept. It is a mess!!!
Remember quality is your responsibility.
NEVER ask permission to do job correctly.
When monitoring complexity track trends not points. (exact values are meaningless but tends are meaningful).
Brendan Forster - 10 Things Ive Learned From Doing OSS
- Read other people code
- there are lots of ways to collaborate. e.g donations, Documentation
- Scratch your own itch
if you have considerable code you should share it, consider your contract.
- you are not your code
- words are hard
- be familiar with software licenses
- friction sucks, Make it easy for me to hack on your code.
- “done” vs “dead”
Enrico Campidoglio - Why no code reviews
Problem 1 is about Ego. EGO=1/Knowledge. The less the knowledge the greater the EGO.
Problem 2: you == your code
Problem 3: Fear of making mistakes. Fear of let colleges see my mistakes
Problem 4: Culture o(lack of it) “you are wrong”…. See Ego!
Do not review the person. REview the code. Do not speak in absolute. Fools are speaking absolute.
Time: it need time.
NB: Good diagram about productivity of a geek
What do you get from code review?
- code quality goes up
- bugs go down, especially fund during code review,.
- sharing knowledge
When you know that someone is watching you will take care of you code more. “The Big Brother EEffect”
It can eager ambition because yo want to impress your programmer fellow.
Give constructive feedback.
In a project everyone is in the same boat. (Sometimes I do not have the feeling - Otto - But we have to trust people that they are doing their bests)
Kind of review
- Formal code review. Most bugs found
- Over the shoulder code review. Less former. Less bug to be found
- Async code review. Happen async to the code written. “distributed”. IT happens offline. Good compromise in nr of bugs found and time.
Tools: - Gerrit, Diffy
- review small portions of code at a time. One commit one logical change.
- keep and follow a checklist
- keep under 1 hour
Aral Balkan - Free is a Lie
Interesting and shocking conspiracy theory how Google/Facebook is ruling our life and data…