Александров Александр Леонидович
Economics of software testing, v.2.0
The continuation of Alexander Alexandrov's talk "Economics of software testing, v.2.0" where we will extend the field for discussions and discuss the issues of the economics of testing with the audience.
The main points of the talk:
- Exhaustive testing is not possible.
- Testing is as well an economic activity.
- Version 1.0, or What need to do. We work within a plan and evaluation, when a lack of resources, we ask to add them (extensive approach).
- Version 2.0, or How need to do. We work within a plan and assessment as efficiently as possible (intensive approach).
The round table will be devoted to the discussion on how to proceed to version 2.0, analysing the following tasks and using reasonable labour costs efficiently, analysing:
- What the test cases we need (do we really need them?).
- What the test data we need.
- What testers we need - a few strong ones (but expensive) or many of weak (but inexpensive).
- What kind of automation we need (do we need it at all), what tools, coverage.
Economics of Software Testing. Version 1.0
It is well known that exhaustive software testing is impossible. However, there are statements about testing completion, although testing can’t be completed, and acceptance testing criteria are economic, rather technological. So testing differs from, for example, development.
The message consider in detail Version 1.0, the main activities are:
- We build restrictions and plan activities
- We estimate the labour costs for testing
- We justify the assessment, show its realism and effectiveness
- We work in accordance with the plan and evaluation
The message also briefly describes main Version 2.0 features - use reasonable work effort effectively by analysing:
- What test cases are needed (and whether they are needed)
- What test data are needed
- What testers are needed - little strong (but expensive) or many weak (but inexpensive)
- What kind of automation (and if at all) is needed, tools, coverage
Investments in labour costs are still returning (ROI), but the goal is efficiency (maximum of benefits)
TEST-manager or test-MANAGER - solving incoming tasks
- Выполнять тестирование проекта с учетом специфики доменной области
- Обучить команду навыкам поиска, занесения и управления дефектами (defect management process)
- Обучить команду тест-дизайну
- Разработать тестовую стратегию основного продукта и независимых стримов продукта
- Провести оценку трудозатрат на тестирование
- Построить план-график работ по тестированию, распределять обязанности (включая бекапирование ключевых ресурсов)
- Мониторить выполнение работ по тестированию
- Готовить отчеты по активностям тестирования и качеству продукта
- Нанимать и обучать тестировщиков (в том числе и при дефиците ресурсов)
- Разрешать сложные ситуации(некачественный тест-дизайн, отклоненные дефекты, конфликты с другими экспертизами)
Round table discussion: TEST-manager or test-MANAGER?
There were many replies from listeners during the talk of Aleksandr Aleksandrov "Current State vs. Common sence" on SQA Days-20 in Minsk, like "Everything you talk is test manager's business", and there were a long after-speech discussion in lobby.
We have found out, that role of test managers is understood differently by all listeners.
During this round table, we are going to discuss mentioned problem and arise auditory, to get answers / thoughts / ideas / opinions for topics we propose to discuss.
- Could you learn to become test manager (and testing)?
- What would (and wouldn't) test manager do on the project?
- Can we work without test manager (and test management)?
- What would you do if project manager rules everything, and testers are not always agree with him?
- Does manager ought to "move away" from testing, and if ought to, how to understand, how much?
- Does manager ought to be "a barrier" for direct communication between testers and "outside world, and how much?
Current State vs. Common Sense
In 1997, Brian Marick wrote "The classical testing mistakes".
In 2009, I conducted an analysis of the state of the mistakes.
The trend seemed encouraging.
Now I have decided to do the same, based on my messagen on SQA Days conferences.
I decided to collect all and understand current trend of topics proposed by myself on the conferences and undersand today's progress of testing as the engineering discipline.