The KAIROS Evaluation Software Suite was developed by NIST in support of evaluation of DARPA's Program on Knowledge Directed Artificial Intelligence Reasoning Over Schemas (KAIROS). Some of the capabilities of this software include:* calculating a variety of metrics and scores indicative of performance of individual KAIROS systems* processing and format conversion of KAIROS system output, data annotations, and human assessment results* analyzing metrics, scores, and assessment results* generating statistics and charts summarizing these results
About this Dataset
Title | KAIROS Evaluation Software |
---|---|
Description | The KAIROS Evaluation Software Suite was developed by NIST in support of evaluation of DARPA's Program on Knowledge Directed Artificial Intelligence Reasoning Over Schemas (KAIROS). Some of the capabilities of this software include:* calculating a variety of metrics and scores indicative of performance of individual KAIROS systems* processing and format conversion of KAIROS system output, data annotations, and human assessment results* analyzing metrics, scores, and assessment results* generating statistics and charts summarizing these results |
Modified | 2021-12-19 00:00:00 |
Publisher Name | National Institute of Standards and Technology |
Contact | mailto:[email protected] |
Keywords | Evaluation of AI systems , Artificial Intelligence (AI) Explainability , Human Language Technology (HLT) , Multimodal and Multilingual Information Retrieval (IR) |
{ "identifier": "ark:\/88434\/mds2-2526", "accessLevel": "public", "contactPoint": { "hasEmail": "mailto:[email protected]", "fn": "Oleg Aulov" }, "programCode": [ "006:045" ], "landingPage": "https:\/\/data.nist.gov\/od\/id\/mds2-2526", "title": "KAIROS Evaluation Software", "description": "The KAIROS Evaluation Software Suite was developed by NIST in support of evaluation of DARPA's Program on Knowledge Directed Artificial Intelligence Reasoning Over Schemas (KAIROS). Some of the capabilities of this software include:* calculating a variety of metrics and scores indicative of performance of individual KAIROS systems* processing and format conversion of KAIROS system output, data annotations, and human assessment results* analyzing metrics, scores, and assessment results* generating statistics and charts summarizing these results", "language": [ "en" ], "distribution": "", "bureauCode": [ "006:55" ], "modified": "2021-12-19 00:00:00", "publisher": { "@type": "org:Organization", "name": "National Institute of Standards and Technology" }, "theme": [ "Information Technology:Data and informatics" ], "keyword": [ "Evaluation of AI systems", "Artificial Intelligence (AI) Explainability", "Human Language Technology (HLT)", "Multimodal and Multilingual Information Retrieval (IR)" ] }