ACADEMIC ABSTRACT

Researchers increasingly use unstructured text data to construct quantitative variables for analysis. This goal has traditionally been achieved using keyword-based approaches, which require researchers to specify a dictionary of keywords mapped to the theoretical concepts of interest. However, recent machine learning (ML) tools for text classification and natural language processing can be used to construct quantitative variables and to classify unstructured text documents. In this paper we demonstrate how to employ ML tools for this purpose and discuss one application for identifying artificial intelligence (AI) technologies in patents. We compare and contrast various ML methods with the keyword-based approach, demonstrating the advantages of the ML approach. We also leverage the classification outcomes generated by ML models to demonstrate general patterns of AI technology development.