Performance Testing

Case Study:

Hi, we have a test environment that is dedicated for performance testing. Has anyone any recommendation about how to manage the use of this environment? (some tests require to have exclusive access to it while running ). Shall we communicate via slack everytime somebody plans to use the env. or shall we do it more official?

Recommendation:

It depends how much control you need - at the extreme end you could disable all access and only enable it when someone requests it, or you could use a calendar booking system, or just notify by slack it’s in use, etc.

-- Alan Giles

Recommendation:

we control access via Jenkins - we have a dedicated single Jenkins executor so two things can’t deploy and test on there at the same time

-- Andy Frith

Recommendation:

How is your test env setup? If it's a VM, maybe you could spin it up/down as needed, and allow for multiple when you want multiple tests going on. I've done this on several projects, and it's a huge cost saver.

-- Max Saperstone

Recommendation:

this depends on a few things:

• how easy/fast/elastic your environment spin-up is.

• how complex is the system under test (i.e. single service or highly integrated distributed system)

• how long and hard do the tests go (i.e. a massive long load test vs. a few user performance checks)

for single services like APIs, consider spinning it up only when needed so there's less config drift and the process is codified.for large end-to-end systems, you'll probably want a flagging mechanism that everything can be considerate of, so that. even a Google calendar can be queried via API to verify if a test is scheduled and no maintenance or other load tests should use the system.if your org uses a cloud to deploy the system to, provided most dependencies also can be spun up in that cloud, you may want to tap into the release engineers' perspective on how hard it would be for more elaborate systems to be deployed dynamically. then you can tear them down afterwards.

-- Paul Bruce

Recommendation:

You can have a look over the Performance Center from Micro focus, it will help you to manage better the calendar and to know where test infra are in use when one of your teammate wants to kick off a perf test and so on. We used it in a daily basis and found it very useful from this point of view.

-- Marius PT