Previous - Next
This is the old United Nations University website. Visit the new site at http://unu.edu
At several points in the phased evolution of the technology development and transfer process, it will be necessary to assess and evaluate potential identified technologies and a hypothetical development and transfer process. Therefore, it is necessary to have criteria for evaluating relative appropriateness of various technology development or transfer strategies in order to determine appropriately meritorious technologies and associated development and transfer strategies. There are many factors that need to be considered in doing this. Large-scale technology development consumes financial and other resources, often for a significant time period. It is invariably necessary to recognize that the benefits of developing one particular technology strategy alternative must be weighed against the costs of foregoing other opportunities.
There are a number of issues to be resolved through the evaluation efforts accomplished as part of the systems management of technology development and transfer. These include:
- determining an appropriate specific process to use for the identification and evaluation of potential technologies for development and/or transfer;
- identifying the groups that should be involved in this identification and evaluation process;
- identifying the criteria that will be used to determine length and type of support;
- identification of appropriate criteria to determine transferability of the technology to full-scale operational deployment status or termination.
Each of these relates to the criteria that will actually be used for evaluating emerging technology development strategies. I have discussed many of these criteria in our efforts this far. Dutton and Crowe [12] provide an excellent summary of many appropriate evaluation attributes:
(1) Technological merit
(a) Technological objectives and significance
(b) Breadth of interest of strategy
(c) Potential for new discoveries and understandings
(d) Uniqueness of proposed development strategy(2) Social benefits
(a) Contribution to improvement of the human condition
(b) Contribution to national pride and prestige
(c) Contribution to international understanding(3) Programmatic (management) issues
(a) Feasibility and readiness for development
(b) Technological logistics and infrastructure
(c) Technological community commitment and readiness
(d) Institutional infrastructure and implications
(e) International involvement
(f) Cost of the proposed strategy
I have rephrased these as attributes for a multiple attribute evaluation effort. These attributes may be viewed in several ways. For example, they can also be seen as ingredients at the various gateways for the development of emerging engineering technologies shown in figure 3. In addition to obtaining an evaluation of proposed technologies and associated development and/or transfer strategies, an appropriate approach to evaluation should also allow for full exploration of the needed functions to insure satisfactory development of appropriate technologies. To do this, people are needed who fill the roles of [41]:
- Idea generators that contribute ideas from technology push or market pull considerations to extend the ultimate potential of the emerging technology under development;
- Idea exploiters, or innovators or technology development champions or research entrepreneurs, who take research ideas and attempt to get them fully explored, supported, and adopted;
- Management leadership, or business leaders, who see to it that the various planning, scheduling, monitoring, and control functions are carried out effectively;
- Information (and knowledge) gatekeepers who provide informed wisdom to the parties at interest as to the emerging engineering technology development relative to contemporary realities that effect technology, capital, manufacturing, standards, and market potential; and
- Sponsors who are not directly involved with the development strategy, in order to insure objectivity, and who provide leadership and resources from the very highest levels to enable development of the technology or to restrict it when it proves cost-ineffective.
These needs should also be translated into attributes that can be used to measure the success of a particular technology and development strategy.
The assessment and evaluation of alternative technologies proposed for development may be approached through the application of formal decision theoretic methods. A major objective in technology assessment and evaluation is forecasting the potential costs and the resulting effectiveness of technology development or transfer. It is desirable to minimize the error associated with this prediction. If a set of inductive beliefs do not conform to those associated with the use of the probability calculus, then the expected error can always be reduced by modifying those belief values to conform to the calculus. Multi-attribute utility models are generally additive. They calculate utilities of an alternative or outcome by a weighted addition of the utility values of the alternative or outcome across the multiple attributes. The reasons that support using these MAUT models are that the necessary and sufficient conditions for these models to hold are well known. Further, the very important notion of value independence results. A cost effectiveness model [53, 58] for evaluating alternatives may be utilized.
The broad goals of cost-benefit analysis are to provide procedures for the estimation and evaluation of the benefits and costs associated with alternative courses of action. In many cases, it will not be possible or appropriate to obtain a completely economic evaluation of the benefits of proposed courses of action. In this case, the word "benefit" is replaced by the multi-attribute term "effectiveness."
Identification and quantification of the benefits and costs of possible alternative courses of action for technology development and/or transfer is a difficult task. It is generally not as difficult perhaps as formulation of the issue and identification of the alternatives themselves, but it is still not easy. Here I use the word benefits to mean the possible effects of a project. These include the totality of both positive and negative benefits, or disbenefits. We must first identify benefits, and then we should quantify them by assigning a value to them. Many benefits (and disbenefits) will be intangible and will occur to differing groups or individuals in differing amounts. Problems with intangibles may be especially difficult in the public sector, where agencies are designed primarily to deliver services or public goods rather than products for individual consumption. A major goal of a private sector organization is profit maximization, and it is relatively easier to measure profit as a benefit. Often there will be a variety of reasons why people will be uncomfortable with providing a strict economic measure for benefits. The word "effectiveness" is often used instead of benefit when a strictly economic valuation is not needed. When effectiveness is substituted for benefit we obtain a cost-effectiveness analysis. The benefits of a public service are much more difficult to define because they are intangible or indivisible (or both).
The political environment of many public-sector efforts further complicates the task, and variables other than those associated with efficiency economy, and equity should be measured. This suggests a multiple perspective approach to effectiveness. Among the many possible perspectives that need to be considered are economic, technical, legal, social, and political.
In cost-effectiveness analysis, we desire to rank projects in terms of economic costs, and in terms of effectiveness. The reason for this is that there are non-commensurate attributes of a project. Certainly we would wish to eliminate conspicuously inferior projects, that is to say projects that are more expensive and less effective than other projects, from consideration for selection. Beyond this, a cost-effectiveness analysis does not specify which of several projects is "best." This can be accomplished if one is willing to trade off cost for effectiveness, such as to obtain a "scaler performance index." It can be done by considering cost as one of the attributes in the effectiveness evaluation approach selected.
The effectiveness of an alternative is the degree to which that alternative is perceived as satisfying identified objectives. The effectiveness assessment approach described here provides an explicit procedure for the translation of quantitative evaluation of alternatives when the impacts of the alternatives are described by multiple attributes. This is accomplished by identifying and organizing the attributes of event outcomes (or alternatives, if there are no probabilistic uncertainties that influence the outcome that will result from alternative selection) into a tree-type hierarchy of attributes that is used together with measures of effectiveness to compare alternative technologies and development strategies as a basis for choice-making. Generally, an effectiveness assessment study involves a number of major analytical steps and is illustrated in figure 12. The final results of a cost-effectiveness assessment are used for comparison, ranking, and prioritization of the identified technology development alternatives according to effectiveness. This effectiveness assessment can be very useful for interpreting and evaluating the results of an analysis effort. In order to use the approach, we need a set of attribute of objective measures, information on the relative importance of attributes or objectives, and sufficient knowledge about project alternative scores and their outcomes to be able to assign effectiveness scores to the attribute measures that characterize the impacts of each outcome.
7. Information technology perspectives
While information technology does indeed enable better design of systems and existing organizations, it also enables the design of fundamentally new organizations and systems. Thus, efforts in this area include proactivity in the sense of being aware of future technological, organizational, and human concerns so as to support graceful evolution over time to new information technology-based services. Among these would be improved access to knowledge of all types.
Figure 12 Steps in formal evaluation of technology cost and operational effectiveness
The initial efforts at provision of information technology-based systems concerned support through more advanced information technology-based hardware and software. Some time ago it was recognized that support could be provided not only to individuals in accomplishing such tasks as report preparation but that the ubiquitous computer could provide support for groups in answering queries of a what if nature with an if then response. This led, two or three decades ago, to the development of support through management information systems (MIS). These systems have become quite powerful today and are used for a variety of purposes, such as scheduling airplane flights and booking passenger seats on them, and registering university students in classes.
As management information systems began to proliferate, it soon became recognized that at least two difficulties remained. While the MIS was very capable of providing support for organizing data and information, it did not necessarily provide much support for human judgement and choice activities. Many such activities need support. They range from providing support in assessing situations, such as to better detect issues or faults, and to support diagnosis in order to enable the identification of likely causative or influencing factors. Nor did the classical MIS provide support for decision-related issues that involve selection of alternatives that have multiple and non-commensurate attributes. This capability was provided by support through judgement and decision support systems. These systems involved linking the database management systems (DBMS) so common in the MIS era with the model base management system (MBMS) capability made possible through advances in operations research and artificial intelligence with the visualization and interactive presentation capability made possible through dialogue generation and management systems (DGMS). The resulting systems are generally known as decision support systems (DSS) [57]. These systems provide needed support for information processing by individuals and organizations.
An additional difficulty is that it has become essentially impossible to cope with the plethora of new information technology-based support systems. The major reason for this is the lack of systems integration across the large variety of such products and services. This has led to the identification of an additional role for information technology professionals, one involving support through information systems integration engineering. An information systems integration engineer is responsible for overall systems management, including configuration management, to insure that diverse products and services are identified and assembled into total and integrated solutions to information systems issues of large scale and scope. There are many contemporary technological issues here. There is a need for what are often called "open systems architectures," or open systems environments, that provide for such needs as inter-operability of applications software across a variety of heterogeneous hardware and software platforms. The key idea here is the notion of open, or public, a notion that is intended to produce consensus based developments that will ameliorate difficulties associated with lack of standards and the presence of proprietary interfaces, services, and protocols.
Figure 13 Evolution of information technology systems over time and development effort
When brought to fruition, these information systems integration developments, and the associated open systems environments, will enable more efficient and effective configuration managements. This will result in the development of information technology solutions in enabling existing organizations to function "better." However, there is also a need for "better" organizational designs. These needs exist in a variety of areas that range from more efficient and effective enterprise management, to more efficient and effective education of students in universities, to more efficient and effective manufacturing processes. It is in this area that contemporary proactive information technology developments promise the greatest pay-off. This is what we attempt to illustrate in figure 13. Through appropriate development efforts, and aided by effective approaches to operational and strategic quality assurance and management, systems integration, and standards, it should be possible to achieve the many objectives for development through systems engineering illustrated in figure 14.
It is important to consider the many impacts that various usages of contemporary information technology have on the environment. Huber [23], in an especially insightful article, identifies 13 propositions that relate to the effect of information technologies on organizations and on associated organizational situation assessment and decision-making.
The first three of the propositions deal with the effects on subunit structure and processes. Huber states that the use of information technology will lead to: a larger number and variety of people participating as information sources in the making of decisions; decreases in the number and variety of people comprising the traditional face-to-face decision unit; and less organizational time being absorbed by decision-related meetings. Six propositions deal with the organization as a whole. He indicates that the use of information technology in a given organization will lead to: a more uniform distribution, across organizational levels, of the probability that a specific organizational level will make a particular decision; a greater variation across organizations in the levels at which a particular type of decision is made; a reduction in the number of organizational levels involved in authorizing proposed organizational actions; fewer intermediate nodes within organizational information processing networks; more frequent development and use of computerized databases as components of organizational memory; and more frequent development and use of in-house expert systems as components of organizational memories.
Huber presents two propositions that deal with situation assessment. He states that the use of information technology will lead to more rapid and more accurate identification of problems and opportunities and to organizational situation assessment that is more accurate, comprehensive, timely, and available. Finally, he presents three propositions that deal with information technology effects on decision-making. It is postulated that the use of information technology will lead to higher quality decisions, to a reduction in the time required to authorize proposed organizational actions, and that the time required to make decisions will be reduced. On the basis of these 14 propositions, Huber-identifies four elements or constructs and obtains a causal structural model as shown in figure 15. This is a fitting concluding picture for this paper, as it does indicate the considerable role that modern information technology-based systems can be expected to have in present and future organizational environments. This will involve: generating the context for information technology through integrating and aligning the strategies for the organization and for information technology developments; engineering the design of an information technology-based system through involvement of users in all life-cycle phases of systems design; and fielding the system in a manner that provides maximum benefits to organizations and humans, including their access to science and technology. This will require very careful attention to such critical ingredients as software systems engineering [59] and related efforts that involve information processing in systems and organizations [56].
Figure 14 Relations between systems management and total quality achievement
Figure 15 Conceptual model of effects of information technology on access to science and technology in organizations and nations
This paper provides a framework for the use of the methods and methodologies of information technology and system engineering, and the application of these to enhance ultimate development and/or transfer of information technologies, especially in developing nations. A major potential use for this is in enhancing access to science and technology for societal improvement. To do this effectively will require much attention to international issues relating to telecommunications [35] and other information technologies. It will require much attention to national and international prosperity concerns affecting the competitive advantage of nations [37]. It will necessarily involve a plethora of considerations concerning the international transfer, or infusion, of technology [44]. I have discussed the many systems engineering considerations needed to beneficially address several problem areas that affect the many groups and issues involved. I have outlined, in general terms, the requirements that such a support system should fulfil.
1. Allen, T.J. (1977). Managing the Flow of Technology. Cambridge, Mass.: MIT Press.
2. Andriole, S.J., and S.M. Halpin, eds. (1991). Information Technology for Command and Control. New York: IEEE Press.
3. Anthony, R.N. (1988). The Management Control Function. Cambridge, Mass.: Harvard Business School Press.
4. Bainbridge, L., and S.A. Ruiz-Quintanilla, eds. (1989). Developing Skills with Information Technology. Chichester, UK: John Wiley and Sons.
5. Beutel, R.A. (1991). Contracting for Computer Systems Integration. Charlottesville, Va.: Michie Co.
6. Cargill, C.F. (1989). Information Technology Standardizaton: Theory, Processes and Organizations. Bedford, Mass.: Digital Press.
7. Clemons, E.K. "Evaluation of Strategic Investments in Information Technology" (1991). Communications of the ACM 34 (1): 22-36.
8. Cohen, M.D., and J.G. March, eds. (1991). "Organizational Learning: Papers in Honor of and by James G. March" Organization Science 2 (1): 1-147.
9. Cohen, S.S., and J. Zysman (1988). "Manufacturing Innovation and American Industrial Competitiveness." Science 239 (4844): 1110-1115.
10. Computer Science and Telecommunications Board of the USA National Academy of Engineering (1991). Keeping the U.S. Computer Industry Competitive: Systems Integration. Washington, D.C.: National Academy Press.
11. Detrouzos, M.L., R.K. Lester, and R.M. Solow (1989). Made in America: Regaining the Productive Edge. Cambridge, Mass.: MIT Press.
12. Dutton, J.A., and L. Crowe (1988). "Setting Priorities among Scientific Intiatives." American Scientist 76 (November): 599-603.
13. Edosomwan, J.A. (1989). Integrating Innovation and Technology Management New York: John Wiley and Sons.
14. Fasser, Y., and D. Brettner (1992). Process Improvements in the Electronics Industry. New York: John Wiley and Sons.
15. Florida, R., and M. Kenney (1990). The Break-through Illusion: Corporate America's Failure to Move from Innovation to Mass Production. New York: Basic Books.
16. Frey, D.N. (1989). "R&D to the Marketplace: A New Paradigm?" The Bridge 19: 16-20.
17. Gallo, T.E. (1988). Strategic Information Management Planning. Englewood Cliffs, N.J.: Prentice Hall.
18. Goodman, P.S., and L.S. Sproull, eds. (1990). Technology and Organizations. San Francisco, Calif.: Jossey Bass Inc.
19. Harvard Business Review (1990). Revolution in Real Time: Managing Information Technology in the 1990s. Cambridge, Mass.: Harvard Business School Press.
20. Hayes, R.H., S.C. Wheelwright, and K.B. Clark (1988). Dynamic Manufacturing: Creating the Learning Organization. New York: Free Press.
21. Heylighen, F. (1992). "A Cognitive Systemic Reconstruction of Maslow's Theory of SelfActualization." Behavioral Science 37 (1): 39-58.
22. Horowitch, M., and C.K. Prahalad (1976). "Managing Technolgical Innovation: Three Idea Modes. " Sloan Management Review 17 (2).
23. Huber, G.P. (1990). "A Theory of the Effects of Advanced Information Technologies on Organizational Design, Intelligence, and Decision Making." Academy of Management Review 15 (1): 47-71.
24. Kaplan, R.S. (1989). "Management Accounting for Advanced Technological Environments." Science 245 (25 August 1989): 819- 823.
25. Keen, P.G.W. (1991). Shaping the Future: Business Design through Information Technology. Cambridge, Mass.: Harvard Business School Press.
26. Lehr, L.W. (1979). "Stimulating Technological Innovation: The Role of Top Management. " Research Management 22 (6).
27. Madnick, S.E. (1987). The Strategic Use of Information Technology. New York: Oxford University Press.
28. Malone, T.W., and S.A. Smith (1988). "Modeling the Performance of Organizational Structues." Operations Research 36 (3): 421-436.
29. March, J.G., and J.P. Olsen (1989). Rediscovering Institutions: The Organizational Basis of Politics. New York: Free Press.
30. National Research Council (1987). Management of Technology: The Hidden Competitive Advantage. Washington, D.C.: National Academy Press.
31. Nilsson, E.G., E.K. Nordhagen, and G. Oftedal (1990). "Aspects of Systems Integration." In: Proceedings of the First International Conference on Systems Integration (April 1990). IEEE Computer Society Press, pp.434-443.
32. Orlikowski, W.J. (1991). "Integrated Information Environment or Matrix of Control? The Contradictory Implications of Information Technology." Accounting, Management and Information Technologies 1(1): 9-42.
33. Orlikowski, W.J., and J.J. Baroudi (1991). "Studying Information Technology in Organizations: Research Approaches and Assumptions." Information Systems Research 2 (1): 1-28.
34. Parker, M.M., R.J. Benson, and H.E. Trainor (1988). Information Economics: Linking Business Performance to Information Technology. Englewood Cliffs, N.J.: Prentice Hall.
35. Pool' I.S. (1990). Telecommunications Without Boundaries: On Telecommunications in a Global Age. Cambridge Mass.: Harvard University Press.
36. Porter, A.L., A.T. Roper, T.W. Mason, F.A. Rossini, and J. Banks (1991). Forecasting and Management of Technology. New York: John Wiley and Sons.
37. Porter, M.E. (1990). The Competitive Advantage of Nations. New York: Free Press.
38. Raelin, J.A. (1991). The Clash of Cultures: Managers Managing Professionals. Cambridge, Mass.: Harvard Business School Press.
39. Rasmussen, J., K. Duncan, and J. Leplant, eds. (1987, 1991). New Technology and Human Error. Chichester, UK: John Wiley and Sons. Vol. 1 - 1987, Vol. 2 - 1991.
40. Repo, A.J. (1989). "The Value of Information: Approaches in Economics, Accounting, and Management Science." Journal of the American Society for Information Science 40 (2): 68-85.
41. Roberts, E.B., ed. (1987). Generating Technological Innovation. Oxford: Oxford University Press.
42. Roberts, E.B. (1988). "Managing Invention and Innovation." Research and Technology Management (January 1988): 11-29.
43. Roberts, E.B., and C.A. Berry (1985). "Entering New Businesses: Selecting Strategies for Success." Sloan Management Review 26 (3).
44. Robinson, R.D. (1988). The International Transfer of Technology. Cambridge, Mass.: Ballinger.
45. Rockart, J.F., and C.V. Bullen, eds. (1986). The Rise of Managerial Computing. Homewood, III.: Dow Jones-lrwin.
46. Rockart, J.F., and D.W. DeLong (1988). Executive Support Systems: The Emergence of Top Management Computer Use. Homewood, Ill.: Dow Jones-lrwin.
47. Rossak, W., and P.A. Ng (1991). "Some Thoughts on Systems Integration: A Conceptual Framework." Journal of Systems integration 1(1): 97--114.
48. Rossak, W., and S.M. Prasad (1991). "Integration Architectures: A Framework for System Integration Decisions." In: Proceedings 1991 IEEE Systems, Man and Cybernetics Conference, Charlottesville, Va.: October 1991, pp. 545-550.
49. Rouse, W.B. (1991). Design for Success: A Human-Centered Approach to Designing Successful Products and Systems. New York: John Wiley and Sons.
50. Rouse, W.B. (1992). Strategies for innovation: Creating Successful Products, Systems, and Organizations. New York: John Wiley and Sons.
51. Roussel, P.A., K.N. Saad, and T.J. Erickson (1991). Third Generation R&D: Managing the Link to Corporate Strategy. Cambridge, Mass.: Harvard Business School Press.
52. Rubenstein, A.H., A.K. Chakrabarti, R.D. O'Keefe, W.E. Souder, and H.C. Young (1976). "Factors Influencing Innovation Success at the Project Level." Research Management 19 (3): 15-20.
53. Sage, A.P. (1983). Economic Systems Analysis: Microeconomics for Systems Engineering, Engineering Management, and Project Selection. North Holland: Elsevier.
54. Sage, A.P. (1987). "Knowledge Transfer: Innovative Roles for Information Engineering Education." IEEE Transactions on Systems, Man and Cybernetics 17 (5): 725-728.
55. Sage, A.P. (1989). "Systems Management of Emerging Technologies." Information and Decision Technologies 15 (4): 307-325.
56. Sage, A.P., ed. (1990). Information Processing in Systems and Organizations. Oxford: Pergamon Press.
57. Sage, A.P. (1991). Decision Support Systems Engineering. New York: John Wiley and Sons.
58. Sage, A.P. (1992). Systems Engineering. New York: John Wiley and Sons.
59. Sage, A.P., and J.D. Palmer (1990). Software Systems Engineering. New York: John Wiley and Sons.
60. Souder, W.E. (1989). "Improving Productivity Through Technology Push." Research/Technology Management 32 (2): 19-24.
61. Strassmann, P.A. (1985). Information Payoff: The Transformation of Work in the Electronic Age. New York: Free Press.
62. Strassmann, P.A. (1990). The Business Value of Computers. New Canaan, Conn.: Information Economics Press.
63. Thurow, L. (1992). Head to Head: The Coming Economic Battle Among Japan, Europe, and America. New York: William Morrow and Company.
64. Tinnitello, P.C., ed. (1989). Handbook of Systems Management, Development and Support. Boston, Mass.: Auerbach.
65. Walton, R.E. (1988). Up and Running: Integrating Information Technology and the Organization. Cambridge, Mass.: Harvard Business School Press.
66. Zuboff, S. (1988). In the Age of the Smart Machine: The Future of Work and Power. New York: Basic Books Inc.
A role for the UNU/IIST: Developing countries' access to new information technologies
Abstract
1.
Part 1: UNU/IIST
2. Part 2: Advanced applications
3. Part 3: Advanced technologies
4.
Conclusion
References
Dines Bjørner, Presented by Zhou Chao Chen
The paper has three parts: first I present the UNU/IIST, then I relate its work to the issues raised during the Kyoto Symposium, and finally I illustrate some additional possibilities. The three parts define complementary access approaches to information technology - axes that the UNU/IIST wishes to offer and pursue. The UNU/IIST offers state-of-the-art training and dissemination, projects and research in the software technology aspects of information technology.
