Month: February 2008

fijiaaron

Modeling Products and Projects

I think many PM tools suffer from the problem of conflating multiple domain models, most likely in the attempt to shoehorm them into the same tool The simple PM tools I talked about (and reviewed previously) suffer from the additional problem oversimplifying, and provide inadequate domain elements to really model the process.

I’ve come up with two (mostly) parallel models that cover the main process domains of product development. It helps me to think in terms of products and projects.

I’m not the first to make the distinction, and certainly not the first to have conflated them. Separating them and managing both in parallel helps me. I use the following two hierarchies:

Product -> Release -> Features -> Requirements

Project -> Milestone -> Tasks -> Work

Definitions and notes:

  • A product is the artifact that exists as a result of the development process.
  • A release is an iteration of the product development process tied to one or more products.
  • A feature is the implementation of one or more requirements.
    • Features may consist of sub-features or components.
  • A requirement is the definition of a business or functional needs.
    • A project is an instance of that development process, the goal of which is to deliver a product.
      • A project may have one more milestones.
      • A project has a definite start and ending
    • A milestone is an enumeration of tasks to be completed in a certain timeframe.
      • A decision needs to be made whether the timeframe or the task list determines the milestone.
        This exercise is called “scoping.”
      • The list of features defines the scope of a product.
    • A task is an element of work that needs to be accomplished.
      • Tasks may have one or more sub-tasks
      • Tasks are assigned to one person
      • Tasks also have dependencies (i.e., other tasks that must be performed first.)
        Dependencies could be an attribute, but there are potentially multiple dependencies.
        (That may just be a flaw in XML-centric thinking. I can’t think of a reason an attribute can’t be a list.)
    • A unit of work is an amount of time spent by one person on a task.
      • A task may have mutliple units of work before being complete.
      • Tasks can track estimated and actual work — measured in units of time — which can be used for time tracking and schedule planning.
        • There should be an initial and a current estimate.
        • The current estimate should equal actual work at the completion of a task

    Notes:

    Scope is the enumeration of features that will be implemented in a release.  Scoping takes into account the time required to complete the tasks required to complete each feature in a release, as well as any additional tasks (e.g., related to testing, deployment, etc.

    A deployment is a release to an environment.  E.g., deployment to test, staging, or production.

    Issues block tasks.  An issue can result in additional tasks.  It can lead to the  documentation of a defect or enhancement.

    Requirements may be separated from features.  (I can sense, based on my guidelines in the post before that my model might not be entirely correct.)  (I think the issue here is that there are mutiple meanings for the word “requirement”.)
    Catchall features such as “security” or “usability” may be used — though I don’t like that practice.
    It makes it too easy to make what should an overarching requirement into a feature that can be cut.
    I think it is better to specify the security or usability requirements for each features (and for the product in general)

    Example:

    The product is a web site. The site will be built in stages. The current release will implement several features, including a shopping cart. The shopping cart has several requirements including “ability to add products from the catalog to cart” and “credit card information will be handled securely.” These requirements may have implementation details. The difference between a business requirement and a functional requirement is whether it it tied to the implementation.

    The project is to develop the website. A milestone may the release of a set of features to production (or test). The tasks for that milestone may include implementing the shopping cart, and fixing certain existing defects.  Tasks required for completing the shopping cart feature might include design, development, testing, and deployment.

    A milestone may indicate a the completion of a stage such as planning, design, development, testing, or deployment.  Or it might mean the completion of certain features.  A release is typically a milestone.

    — I’m getting some conflation here.

    Feedback:

    I’d be curious to know if others have different ways of modeling it, or if they see flaws or advantages in what I’ve outlined.

    fijiaaron

    Thinking about development and related processes

    In my mind, product development is broken down into three stages:  design, development, and deployment.  Project management and quality assurance are supplementary activities.  Project management manages scheduling, budget, resources (people and things) and tasks.  Quality Assurance is  responsible for testing, process, and requirements management.  In general, PM interfaces with “Business”, and QA is concerned about “Customers.”  Operations handles the product after deployment.

    fijiaaron

    Model Hierarchies

    Hierarchy complexity

    Hierarchies should be 4-7 levels deep.

    Three or less is not really a model.  Ten or more is too complex.

    Two levels is really just a tuple.  Three levels is is just grouping of tuples, which doesn’t really require modeling.  Of course there are exceptions.

    Artificial Hierarchies

    You often see models (especially in XML, but also in OOP) where people sense this, and create artificial depth in hierarchies.  For example:

    <product>

    <features>

    <feature>

    <features> is just a useless wrapper around features that tells you “one or more features” may follow.  That’s really just a crutch for the parser.  You could just count up the individual <feature> element underneath <product>.

    If a featurset has attributes that can’t be expressed in the individual features, then its okay. But if it’s only a grouping, then the grouping is really just an instance of a higher level element.

    You also see a base element that is only there to have a base element.  That’s a flaw in XML, but a base element can be good for descriptive naming purposes, or to describe domain level attributes.  I think top level attributes are a bad practice though.

    fijiaaron

    Guidelines for domain modeling

    I think two common mistakes are made in modeling data & processes that can lead to unintended complexity.  The first is conflating two models that should be separate, and the second is oversimplifying models.Conflated model: when what should be two separate models are intermingled, resulting in confusing (forced) associations.  If you have two elements in a hierarchy and you’re not sure which is higher, it could be a sign of conflated models.

    Inadequate model: when the model is too simple, resulting in multiple aspects being crammed into one element — making comparison and isolation difficult.  If an element has too vague a meaning or has muliple contexts, it may be better modeled in separate elements.

    Contexts are a cood indicator of domain areas.  Requiring context to understand an element may hint at two or more models.

    fijiaaron

    Thinking about PM tools

    I’ve been thinking about Project Management tools a lot lately.

    One step I took was to try out a bunch of the new breed of lightweight tools:  Basecamp, GoPlan, ActiveCollab, ProjectPier, Wrike, etc.  I wasn’t really satisfied with any of them.  While lightweight tools are a breath of fresh air for those coming process heavy organizations, I doubt they really satisfy.  The market for simple tools is really for those who’ve never quite figured out how to implement process.  The “some process is better than no process” argument.

    But a spreadsheet beats most of those tools hands down in every aspect except distributed teams.  Sharepoint will usually handle that problem, but so would a web interface that implements the spreadsheet almost exactly.

    I’ve also tried heavy process tools such as the Rational Suite and e-Project (whatever they’re called now.)  Come to think of it, the real weight in these tools is the tools themselves.

    Clearquest’s web interface is miles better than it’s rich client, something I acknowledged a couple years ago (and the rich client hasn’t gotten any worse.)  But it’s still no fun, and you have 10 screens you need to step through to ignore 90% of it.  And integration wit Cleacase is a pain.

    The other tool,  e-Project could probably improve with a facelift, but you end up spending alot of time overriding default behaviors to allow it to be useful at all.  It’s the proverbial webified spreadsheet, only it doesn’t have the spreadsheet functionality.  Some AJAX could really help it out, but it might be too heavyweight, espcially with hundreds of tasks.

    So of course I’m thinking of a middle ground.  You want a simple enough model, but you also want tools for power users too.  You just want them to stay out of the way until needed.  The problem here is you end up bolting stuff on and it looks bolted on.  You can’t start with every feature, but starting with a good model and a reasonable (streamlined) process is important.

    fijiaaron

    ¿Viva Fidel?

    I’ve interrupted my musings to bring you a celebration of Capitalism that is both timely and topical…

    ¿Viva Fidel?

    Is he alive? That’s the question. And has been for the past 6 months or so. But today, there’s evidence he might be, or might not be.

    Being startled from my reverie of hacking Typo3 to work with MySQL 5 by Kelsey’s announcement that Fidel Castro resigned today, she hit on a theme / that we thought might sell t-shirts.

    Check it out on our shared blog Dos Puerqueños (and fight to get “puerqueño” added to wikipedia. It means “roast cuy” which is guinea pig and it’s a Spanish portmaneau of “puerco” meaning pig and “pequeño” meaning small.)

    Or head over to CafePress and get your ¿viva fidel? gear.

    Viva Fidel t-shirtviva fidel t-shirt

    All proceeds will go to support the Audubon Society of Little Havana or OSSO, which a truly great program, and the catalyst that brought us to Ecuador.

    fijiaaron

    The mass of flames that didn’t come

    I was expecting quite a few reponses to my rant the other day, and maybe even a few comments of my framework requirements, but it looks like nobody reads my blog, after all. Boo-hoo. I mean thank goodness. I can once again feel safe in my shell and make stupid sweeping generalizations with impunity.

    But Kelsey wanted me to shill for her, under the illusion that I am another slashdot. It’s a good thing I didn’t set up adwords, it would have never paid for the bandwidth. As controversial or as asinine as I can be, even mentioning open source products and languages disparagingly doesn’t generate more than a half dozen hits.

    The word “Rails” still gets more than “Django” — but the sample is so small it’s statically insignificant. “CakePHP” and “SolarPHP” don’t seem to draw anyone’s attention. “Project management” seems to fare better than “QA” and “testing” combined, but neither can hold a candle to “ImageMagick” and “Calixto Island”, a 1980s text adventure game for the TRS-80 that I mentioned a while back.

    My puny market research shows definitively that I should create a text adventure written as a series of dynamically generated PNGs with the text overlayed in graphics.

    Once I hit on the proper buzzworthy framework and programming language to use, I’ll be off to the races.

    It seems perl and PHP users have equal frustration with installing ImageMagick, so it’ll probably be one of them — especially because I don’t want to jump through the hoops getting it to work with Scala or F#.

    In the meantime… (see next post)

    fijiaaron

    Leveraging existing knowledge and techniques — ideal framework requirement #3

    Too many framework developers decide they’re going to be too clever by half. As a result, writing plugins, extensions, modules, templates, models, views, controllers, classes, procedures, whatever you call them is always a unique experience.

    My rant about ORM (and templating) tools was one part of this. Writing records to a database is not rocket science. But learning the way tools want you to do it sometimes is. I realize there is sometimes complexity that it is trying to hide, but don’t. That’s what I’m talking about with magic.

    It’s okay to use an abstraction layer. It’s even okay to use an ORM if you really feel like it. But I should be able to quickly trace my code’s path and debug what’s happening. Even if (most like) it isn’t your (the framework designer’s) code, it’s mine. At least I won’t spend all day pointing my finger at you.

    But the truth is, chances are, I’m going to have to work around your framework, or customize it, or optimize it, or put in some ugly hack at some point. If you’re too clever by half, I’m going to (incorrectly) assume your framework can’t handle it, and throw it out. The cdbaby story on Oreillynet.com comes to mind.

    Writing records to a database, or alternately using a cache, marking the cache dirty when appropriate, and using either a file-based or in-memory (local or remote) cache isn’t rocket science either. But if I don’t know how you’re doing it, I can’t poke around and figure out that it was my stupid configuration and not your brilliant framework at fault.

    In summary, if I look at (to continue my example) a persistence class, it should look like something I’m familiar with. I’m aware that there are more techniques and idioms than one man can be aware of, but it seems too many people go out of their way to create their own.

    I understand the desire to “do it the way you’ve always wanted to” — that’s a huge part of the open source itch. I’m guilty of it myself. But so many people just have such bad style. Or maybe I just can’t recognise good style when I see it. You probably won’t when you look at my framework and the new way I code.

    Rails created whole new idioms and even rewrote (by dynamic class overloading) some of Ruby’s semantics, but they could do that because there wasn’t really anyone using Ruby at the time, so the syntax, coding style, and language idioms were practically up for grabs, as far as the larger programming community was concerned.

    People (including me) were tolerant of learning Rails idioms (including ActiveRecord) because we were learning a new language anyway. But while some things (like ActiveRecord) had pretty good style; others, like routes, had a smell from the start. It’s often dismissed by humble programmers as “just not getting it” — but lots of people “just didn’t get” EJBs for years before the switch was flipped and suddenly everyone admitted that it was just a bad design from the start.

    Another part of leveraging existing knowledge and techniques is using existing tools. I deliberately left that out, because I don’t always think its best. If you use an existing crappy library, the flavor will spread through your code, but things like loggers, unit tests, etc. have such entrenched methods, that while your annotation-based AOP injected distributed transactional logger is just going to confuse people — oh wait, that sounds like the standard way of doing things in Javaland these days.

    You might be nodding your head and thinking “design patterns” but that’s NOT what I’m talking about. Design patterns, for the most part are something people talk about who want to sound smarter than everyone else. If only people who think they’re smarter than everyone else are going to work on your framework, fine — sprinkle some decorators and anti-singletons, and whatnot around and be sure to use those big words in your documentation. But design patterns are really things like arches and dormer windows, and I don’t think that has anything to do with web frameworks, and come to think of it, I can’t think of anything that arches and dormer windows have in common, except they’re parts of buildings, and neither one of them is really a pattern.

    fijiaaron

    Being obvious about what it’s doing – requirement #2 of the ideal framework

    “What you don’t know can’t hurt you” — but it can’t help you either.

    Like many others, I suppose, I fell in love with Rails because it “Gets out of your way” so well.  At least at first.

    Thanks especially to Ruby’s dynamic method modification and anonymous blocks, Rails hides a lot of the details from the user.  I still can’t think in closures, because I’m an implementation fetishist, but most of the time you don’t need to know how something works.   And moving it out of way only helps you concentrate on what you’re doing.  Until you need to do it some other way.  And then cleverness bites you.

    Like it always does in perl.  Sure, a solid grasp of *how* OOP works makes it possible to understand how OOP in perl works, but you’re spending too much time showing how it works in you code to get the biggest benefit of OOP — simplification.  Which is probably the most underdelivered feature of OOP anyway.

    But a good framework shouldn’t do too much magic.

    For one, it makes people like me — people who like to poke their fingers in the black box — nervous.  There might be something sharp, or something you can break inside that box.  And learning that box’s implementation as well as it’s interface gets to be too big of a burden.

    Well, I suppose I should discipline myself and leave the lid on the box and get my work done…but that wouldn’t be ideal, would it?

    “But what if I need to pop the hood and fix something?” that voice in my head says. I probably overestimate my need (or capability) of fixing (or improving, or modifying) the engine inside that framework box, but there you have it.

    I like CodeIgniter because it’s obvious about what it’s doing.  Maybe 91% of that is their really good documentation, but then maybe that’s 91% of this requirement.

    The rule I take away from this requirement is that you should avoid unnecessary layers, like ORM and templating.  They shouldn’t be central features at least.  People have no trouble dismissing things authentication and authorization as external, but consider their ORM tool or templating system or CMS interface to be the core of their framework.

    Maybe I’m just wrong (or different) in my thinking that authn/authz are core, or maybe it’s just the type of apps I tend to develop.  But I’m willing to accept that their external if you’re willing to accept that your feature is.  Or I guess different frameworks have different “themes” based on their focus — content managment, presentation, persistence, or authorization.

    Another reason to be obvious about what you’re doing is that it makes it easier for us humble demi-hackers to work on.  Less magic means less powerful magicians can provide meaningful contributions.  That’s one area where CPAN shines and PEAR flounders.  PEAR’s coding conventions are a barrier to entry.  CPAN has tests that accomplish the same thing (better) and allow ranking without disallowing contributing.

    Being able to understand how your framework works allows you to work within the framework, rather than trying to work around it.

    When I first started in PHP, I really struggled with session caching, and db connection pooling because I sensed that I wanted something more, and when Java frameworks started to offer tools for this, I was easily swayed, swallowing the camel to get at the fools’ gold, or something like that.  But when I better understood PHP’s shared-nothing architecture, and Apache’s MPM process strategy, I realized that it was a waste of time to try and build that stuff into PHP.

    The final reason being obvious about what you’re doing is because sometimes that voice in my head is right and you do need to open the box, even if it has so fare been a gift from the Gods.  And then you can only hope that you’ll be able to figure out what it’s doing.

    Magic tools are fine when they increase productivity or abstract away implementation details, but for the suspicious and superstitious, being able to see how that magic is performed is important.

    Which leads, unsurprisingly to my next requirement for the ideal framework:

    “Leveraging existing knowledge and techniques”

    fijiaaron

    Getting out of the way – requirement #1 of the ideal framework

    The framework that manages best, manages least”  or something like that.
    Even in the original it didn’t necessarily mean the least amount of government, but that because of the structure, less intervention was required.

    This doesn’t directly boil down to simplicity,  but simplicity is the easiest way to reduce management overhead.  That’s why things like wikis, agile programming, and lightweight project management tools like Basecamp are so popular.  But what do you do when complexity is needed?  Fine grained authorization, detailed workflow, complex views, localization, etc. (More on this later.)

    A good framework should not have complex configuration, obscure conventions, or rigid patterns.

    On the one hand, Rails is very good at this.  You can build sites with Rails without really understand Ruby or Rails.  On the other, it’s very difficult to go outside the Rails box.  Rails has improved, or rather techniques have been learned that make this doable.  For the most common 80% of tasks, though, I’d say Rails sets the standard for getting out of the way.

    Rails’ philosophy of “convention over configuration” works in Rails because for the most part, it gets the intuitive part of convention right.  It also benefits from being an early comer (in at least the current round) among frameworks.  Many imitators who diverge do so at their own peril, first because Rails did an exceptionally good job, and second, because “The Rails Way” (good book by the way, by Obie Fernandez.  Don’t think I’m plugging for him though, I don’t know him, but my uninformed personal opinion of him isn’t positive.) has become a de facto standard of comparison with most other frameworks that came after it.

    ActiveRecord, however, does a great job at first –it was arguably the defining feature that made Rails popular, but then falls down when you want to go outside the class-is-a-table pattern.  Not saying it blocks you, but it gets in the way.  ActiveRecord is so valuable for prototyping new applications, however, it’s almost a must for a good framework to support it.

    I’d argue a good framework should allow things like ActiveRecord, not necessarily support them.  I haven’t yet seen the tool, method, pattern that makes transitioning from ActiveRecord smoothe.

    When convention falls down and configuration is needed, then things can get in the way, and perhaps one of Rails’ stumbling blocks is the advanced beginner has had it so easy so far with sensible defaults, that transitioning to manual configuration is as like hitting a wall as hitting the water at high speed can be.  If you weren’t going so fast to start with, you’d slip right into the configuration.  And after something like Struts or Spring IOC, Rails configuration is like slipping into a jacuzzi after a long traffic jam.

    Similar story with URLs.  Rails’ page-controller default routing is a thing of beauty, and a true aid to site design all around.  But when the page-is-a-controller metaphor breaks down, it’s a pain.  Probably more because you tend to think that way and choose that as the wrong pattern when you shouldn’t have.  Also, without an easily customizable front controller, pages classes tend to have too much “baked” in, and the complexity is in the wrong place.

    The thing I like most about Rails is how well it does “get out of the way”, and the thing I like about CodeIgniter is that it accomplishes it almost as well,  but transparently, and could benefit from some of Rails’ magic.  Which leads to my next requirement for the ideal framework:

    Being obvious about what it’s doing.