The phrase of the day is “User Ability Testing” … quite possibly the best mangling of usability I’ve yet heard.
However, it does bring to mind an interesting thought. At what point does one simply stop adding new features because they’re beyond the ability of 95% of your target audience and no design effort can make them more accessible?
Advanced mode GUI? CLI only? API for the feature? We get to the point eventually where anything to be done must be handled by a developer, or at the least a person trained how to use the tool.
So, does it make sense to have the user environment detect how capable the user is and then selectively enable features at the cost of learning, discoverability, and ease of use? As a person who is happy to spend time figuring things out I don’t mind complex features. Counterpoint: most people don’t know how to program their VCR and in the grand scheme of things this isn’t hard.
I think I just figured out what my point was: make sure you successfully identify your audience or any usability testing is for naught, precluded by insufficient user ability.