testing - How should feature toggles be set in tests run in continuous integration? -
how 1 go testing when using feature toggles? want development computer close production possible. videos watched, feature toggles implemented in way allow people "use" feature (i.e., 0 100 % of users, or selected users, etc.).
to continuous integration correctly, have use same feature toggle settings production servers when comes testing? or better yet, if feature not off on production, make sure it's on when comes running automated tests in build pipeline? end putting feature toggles in testing code, or write tests in new file? when new feature mandatory step in process must occur system tests?
in team of more few people uses feature toggles routinely, it's impractical combinatorial testing of toggles or plan testing of combinations of toggles expected interact. practical strategy testing toggled code has work single toggle without considering states of other toggles. i've seen following process work well:
because move code production possible, when toggle introduced project, new tests written cover toggled code toggle on. because test thoroughly, tests code toggle off exist; tests changed toggle explicitly off. toggled code can developed behind toggle long necessary.
immediately before toggle turned on in production, tests (not tests of toggled code, application's entire test suite) run toggle on. catches breakage due unforeseen interactions other features.
the toggle turned on in production
the toggle removed code (along code active when toggle off) , code deployed production
this process applies both cases toggle hides new functionality (so there no code runs when toggle off) , cases toggle selects between 2 or more versions of code, split test.
to answer couple of specific points:
whether tests of different toggled states go in same file or different file depends on size of toggled feature. if it's small change, it's easiest keep both versions in same file. if it's complete rewrite of major feature, it's easier have 1 or more new test files devoted new state of toggle. number of files affected toggle depends on test architecture. example, in rails project uses rspec , cucumber toggle might require new tests in cucumber features (acceptance/integration tests), routing specs, controller specs, , model specs, and, again, tests of toggle @ each level might in same file or different file depending on size of toggled feature.
by "system tests" think mean manual tests. prefer not have those. instead, automate tests, including acceptance tests, wrote above applies tests. leaving aside, new state of toggle becomes law once temporarily when run tests toggle on before turning on in production, , permanently when remove toggle.
Comments
Post a Comment