The Korea Telecommunications Technology Association (TTA) announced on the 16th that it has granted data quality certification (DQ certification) for the 'LLM harmlessness evaluation data' developed by SelectStar, marking the first such certification in South Korea.
The DQ certification is a system that certifies data content and management systems based on the Framework Act on the Promotion of Data Industry and Use. The TTA has been designated as a data quality certification agency by the Ministry of Science and ICT, providing certification services in three areas: structured data, unstructured data, and data management systems.
The 'LLM harmlessness evaluation data' certified by DQ from SelectStar is benchmark data designed to evaluate the harmlessness of responses generated by large language models (LLMs), capable of assessing key four areas: bias, hate, illegality, and sensitivity. Notably, this data is characterized by its systematic evaluation method using timely assessment items and a rubric reflecting current social issues.
The TTA derived and evaluated criteria that can assess whether the 'LLM harmlessness evaluation data' is suitable as benchmark data related to LLM harmlessness among the total of 24 evaluation items in the unstructured data evaluation system (9 mandatory, 15 optional), concluding that the data meets certification standards.
In particular, the TTA evaluated the clarity and communicative efficacy of the questions used to assess the bias and illegality of LLMs in granting the DQ certification.
Son Seung-hyun, chairman of the TTA, noted, "The quality of benchmark data for large language models (LLMs) is an essential element for AI models to be trusted and safely utilized, and this certification will play an important role in securing the quality of LLM evaluation data, fostering the advancement of the AI industry, and gaining user trust."