Software through the lens of AI

Questions:

1. Given we may be iterating into AI test bots being run by AI test bot managers, with the underpinning assumptions perhaps being checked manually, how do we usefully demonstrate quality and confidence in an application? (If our bots run an enormous amount of tests, how to we extract value from the results to provide *confidence*)

Just like any other system, we have acceptance criteria for AI models as well. This is done using Evaluation Metrics - Accuracy, Precision and Recall.Before training the AI model, the team comes up with acceptable percentages for these 3 metrics and then do the AI model training. After certain level of training they see if the accuracy, precision and recall is within the acceptable percentages, If so, then they decided to release the system in production. For example. If team decides the accuracy should be 95%, Precision should be 85% and Recall should be 90%, then after sufficient training they check to see if the values are within or above these percentages. There are also other metrics like F1-score, area under the curve.

I have used a lot of resources in the past few years, but a good start would be to go through the below ones Machine learning for business professionals - https://www.coursera.org/learn/machine-learning-business-professionals AI for everyone - https://www.coursera.org/learn/ai-for-everyone?utm_source=googlecloud&utm_medium=institutions&utm_campaign=ML_Apr20_newsletter Launching into Machine Learning Course - https://www.coursera.org/learn/launching-machine-learning Andrew NG course. - https://www.youtube.com/watch?list=PLA89DCFA6ADACE599&v=UzxYlbK2c7E The state of AI - https://www.youtube.com/watch?v=NKpuX_yzdYs&feature=emb_logo AI the new electricity - https://www.youtube.com/watch?v=21EiKfQYZXc&feature=emb_logo3.

2. Are there any tools available to help evaluate how negatively biased AI results maybe?

Yes, have IBM 360 fairness toolkit, Google's What-if-tool, Skater, Audit-AI, FairML and other open source libraries4. What was the name of device you mentioned by aws to train your model?

AWS Deep Lens and Microsoft Vision AI developer kit5. Do you think an AI model could be created that could in turn create other AI models? Yes, there are lot of research and work going on this space. There are more useful resources about this in deepmind's research blog/articles - https://deepmind.com/research

In case of more info or details. email me at [e-mail] or visit my website - www.rajsubra.com

-- Raj Subrameyer