The purpose of this research was to design and develop indices of information technology and guideline for self-diagnosis for higher education. The indices of information technology are designed to help stakeholders of higher education ensure what features should be established for their success in information technology. In an effort to achieve the goal, it was presumed that empirically proven evaluation model should be applied to measure and check levels of information technology in developing the indices. The CIPP model was selected for developing the indices because the model was evaluated to have systematic process of making decisions of allocating resources to accomplish a given organization goals. CIPP is an acronym for context, input, process and product. The benefits of this model approach are on the way of mutual interactions between the four components: context, input, process, and product. In this model approach, once the information is collected in terms of the four components, it will show how the resources are going through an organization. Once any changes are happened, the changes will be easily traced on the four components so the evaluation activities have to adapt to meet these changing needs as well as ensuring continuity of focus where appropriate.

Because of these interconnectivities within the components, CIPP is evaluated as a proper framework to design indices of information technology. Two major benefits are delineated. First, CIPP is a better way to have balanced perspectives because an evaluator can measure from input to product. Second, once the evaluator is noticed where more resources are required, he/she can easily reallocate other resources by having the balanced perspective. Along with the benefits, the researchers apply CIPP as a major framework to design and develop the indices. However, we do not apply the context component in developing the indices because the context of information technology is presumed to vary depending on the types of higher education such as public or private.

The researchers added a content related dimension to measure the progression level of information technology while CIPP forms a foundational framework. The dimension consists of five areas: 1) information infra, 2) teaching information, 3) administrative information, 4) research information, and 5) service information. These areas are emerged from the primary functions of higher education: a) teaching, b) research, and c) service. In addition to these three primary functions, information infra and administrative information are added. The information infra is added to take into account of measuring how well an institute establishes hardware resources including human resources to disseminate information technology into an organization. The administrative information is added to evaluate how a supporting system is working well for an organization.

With the combination of five content areas and three different components from CIPP model, the researchers developed more than 50 indices at an initial stage. For the second stage to define the initial items, the researchers screened the items and reduced it to 40 indices. In the second stage, iterative comparison and confirming logical reasoning techniques were applied. Once the initial items were reduced, focus group was formed with experts in the field of information technology in higher education. The reduced initial items were reviewed through a survey form. All the items were measured with 5 points scale for the importance and appropriateness of each item. This method was adjusted from Delphi method. As a result of this focus group survey, the initial items were reduced to 25 indices. However, in an effort to add more balanced perspectives, a public hearing session was held. From the public hearing, two more items were added and one item was dropped from the selected items. Finally, the total number of items to measure the information technology ends with 26 indices. With the given indices, detailed instructions of purpose and evaluation method for each item are provided as a form of guideline. The major goal of guideline is to provide self-initiated diagnostic tool.

The use of indices and guideline for information technology can vary depending upon contextual needs of institute to initiate information technology. There are many items developed to make international comparisons with other nations. These internationally comparable items are originated from official survey items of OCED, and EU.