📋
QASpace-CaseStudies
  • Home
  • Test Strategy
    • Vision of Quality
      • Good Test Coverage
      • ISO 29119 Certification
      • Date Internationalization Format
      • Localization Testing
      • Test Management
      • Creating TestCase
  • How to Test
    • How to write good TestCase?
      • Tips on Testing
      • Pair Testing
      • UserStory to TestCases
      • Optimising the development flow in a Scrum team
      • Rate/Prioritise bug tickets
  • Exploratory Testing
    • Exploratory Testing
      • Creating Test Charters
      • Test Charters
      • Velocity was too high
  • Agile
    • Implementing Scrumban
      • Breakdown Task
      • Common work across teams
      • Sustainable QA process in the organization
  • Philosophy of Testing
    • Brainstorming
  • Metrics
    • Risk Analysis
      • Testing Outsource
      • How to measure Quality?
  • Automation
    • AI-Automation
      • Software through the lens of AI
      • SAP/Salesforce Automation
      • Mobile Automation
      • Solve by automating the GUI?
      • Improve Skill-sets
      • Coding Skills
      • Working in BDD
      • Value of Test Automation
      • UI/API automation asset
      • TDD VS BDD
      • Selenium vs Cypress
      • An important consideration of Test Automation journey
      • Balance Test Automation Development
      • Automation is no longer providing value
      • Define AI in test automation
      • Unique Locators
      • Best Practices as QA, QA Lead, and Automation Engineer
      • Making friends with Imposters
  • Survey/Polls
    • Is QA really a Gatekeeper?
  • Performance
    • Performance Testing
      • Client-Side Performace
Powered by GitBook
On this page

Was this helpful?

  1. Automation
  2. AI-Automation

Software through the lens of AI

PreviousAI-AutomationNextSAP/Salesforce Automation

Last updated 4 years ago

Was this helpful?

Questions:

  1. Given we may be iterating into AI test bots being run by AI test bot managers, with the underpinning assumptions perhaps being checked manually, how do we usefully demonstrate quality and confidence in an application? (If our bots run an enormous amount of tests, how to we extract value from the results to provide *confidence*)

Just like any other system, we have acceptance criteria for AI models as well. This is done using Evaluation Metrics - Accuracy, Precision and Recall.Before training the AI model, the team comes up with acceptable percentages for these 3 metrics and then do the AI model training. After certain level of training they see if the accuracy, precision and recall is within the acceptable percentages, If so, then they decided to release the system in production. For example. If team decides the accuracy should be 95%, Precision should be 85% and Recall should be 90%, then after sufficient training they check to see if the values are within or above these percentages. There are also other metrics like F1-score, area under the curve.

I have used a lot of resources in the past few years, but a good start would be to go through the below ones Machine learning for business professionals - AI for everyone - Launching into Machine Learning Course - Andrew NG course. - The state of AI - AI the new electricity - 3.

2. Are there any tools available to help evaluate how negatively biased AI results maybe?

Yes, have IBM 360 fairness toolkit, Google's What-if-tool, Skater, Audit-AI, FairML and other open source libraries4. What was the name of device you mentioned by aws to train your model?

AWS Deep Lens and Microsoft Vision AI developer kit5. Do you think an AI model could be created that could in turn create other AI models? Yes, there are lot of research and work going on this space. There are more useful resources about this in deepmind's research blog/articles -

In case of more info or details. email me at or visit my website -

-- Raj Subrameyer

https://www.coursera.org/learn/machine-learning-business-professionals
https://www.coursera.org/learn/ai-for-everyone?utm_source=googlecloud&utm_medium=institutions&utm_campaign=ML_Apr20_newsletter
https://www.coursera.org/learn/launching-machine-learning
https://www.youtube.com/watch?list=PLA89DCFA6ADACE599&v=UzxYlbK2c7E
https://www.youtube.com/watch?v=NKpuX_yzdYs&feature=emb_logo
https://www.youtube.com/watch?v=21EiKfQYZXc&feature=emb_logo
https://deepmind.com/research
raj@rajsubra.com
www.rajsubra.com