Gestão & Produção
Gestão & Produção
Artigo Original

Modelo multidimensional para mensurar qualidade em website de e-commerce utilizando a teoria da resposta ao item

Multidimensional model to measure quality in e-commerce websites using item response theory

Rafael Tezza; Antonio Cezar Bornia; Dalton Francisco de Andrade; Pedro Alberto Barbetta

Downloads: 0
Views: 855


Resumo O presente artigo propõe um modelo multidimensional para medir a qualidade de websites comerciais com uso da Teoria da Resposta ao Item (TRI). A qualidade de um website engloba características técnicas (usabilidade/navegabilidade, apresentação da informação e interatividade) e não técnicas (design, estética, apelo visual, confiabilidade, hedonismo, imagem), configurando-se, teoricamente, como contexto multidimensional. As suposições iniciais das dimensões e elaboração dos itens foram baseadas em análise bibliográfica sobre o tema qualidade em websites de e-commerce. Um conjunto de 75 itens foi elaborado e uma amostra de 441 websites de e-commerce dos mais variados setores foi a ele submetido. O tratamento e análise dos dados foram feitos utilizando-se a TRI. Nessa etapa, foram discutidas questões referentes à dimensionalidade e à escolha do modelo mais adequado. Por fim, um modelo multidimensional com quatro dimensões foi ajustado contemplando as dimensões navegabilidade ou condução/orientação do usuário, acessibilidade e confiabilidade do sistema, interatividade e apresentação da informação.


Qualidade em websites, E-commerce, Modelagem, Teoria da Resposta ao Item Multidimensional


Abstract The goal of this article is to propose a multidimensional modeling to measure the quality of commercial websites with the use of Item Response Theory (IRT). The quality of a website encompasses technical characteristics (usability-navigability, presentation of information and interactivity) and non-technical characteristics (design, aesthetics, visual appeal, reliability, hedonism, image), theoretically configuring a multidimensional context. The initial hypothesis of the dimensions and the elaboration of the items were based on a bibliographic analysis about the theme of e-commerce website quality. A set of 75 items was prepared and submitted to a sample of 441 e-commerce websites from a wide variety of sectors. The treatment and analysis of data was conducted using IRT. In this step, questions related to dimensionality and the choice of the most suitable model was discussed. Finally, a multidimensional model with four dimensions was adjusted.


Quality in websites, E-commerce, Modeling, Multidimensional Item Response Theory


Web accessibility checker. 2012.

Ackerman T. A. A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement. 1992;29(1):67-91.

Ackerman T. A. Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education. 1994;7(4):255-78.

Adams R. J., Wilson M., Wang W. C. The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement. 1997;21(1):1-23.

Agarwal R., Venkatesh V. Assessing a firm’s web presence: a heuristic evaluation procedure for the measurement of usability. Information Systems Research. 2002;13(2):168-86.

Aladwani A. M., Palvia P. C. Developing and validating an instrument for measuring user-perceived web quality. Information & Management. 2002;39(6):467-76.

Al-Khalifa H. S. The accessibility of Saudi Arabia government Web sites: an exploratory study. Universal Access in the Information Society. 2010;10(4).

Andrade D. F., Tavares H. R., Valle R. C. Teoria de resposta ao item: conceitos e aplicações. 2000.

Babbie E. The basics of social research.. 2005.

Baker D. L. Advancing E-Government performance in the United States through enhanced usability benchmarks. Government Information Quarterly. 2009;26(1):82-8.

Bangor A., Kortum P. T., Miller J. T. An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction. 2008;24(6):574-94.

Barnes S. J., Vidgen R. T. WebQual: an exploration of web site quality. 2000;1:298-305.

Bartolucci F., Montanari G. E., Pandolfi S. Dimensionality of the latent structure and item selection via latent class multidimensional IRT models. Psychometrika. 2012;77(4):782-802.

Beaton A. E., Allen N. L. Interpreting scales through scale anchoring. Journal of Educational Statistics. 1992;17(2):191-204.

Birnbaum A. Some latent trait models and their use in infering an examiniee’s Ability. Statistical theories of mental test scores. 1968;38:123-40.

Braz C., Seffah A., M’Raihi D. Designing a trade-off between usability and security: a metrics based-model. Lecture Notes in Computer Science. 2007;4663:114-26.

Cai L. FlexMIRTTM version 1.86: a numerical engine for multilevel item factor analysis and test scoring. 2012.

Camilli G. A conceptual analysis of differential item functioning in terms of a multidimensional item response model. Applied Psychological Measurement. 1992;16(2):129-47.

Chalmers R. P. Mirt: a multidimensional item response theory package for the R environment. Journal of Statistical Software. 2012;48(6):1-29.

Cristobal E., Flavian C., Guinaliu M. Perceived e-service quality (PeSQ): measurement validation and effects on consumer satisfaction and web site loyalty. Managing Service Quality. 2007;17(3):317-40.

Day A. A model for monitoring Web site effectiveness. Internet Research: Electronic Networking Applications and Policy. 1997;7(2):1-9.

DeLone W. H., McLean E. R. The DeLone and McLean model of information systems success: a ten-year update. Journal of Management Information Systems. 2003;19(4):9-30.

Deng L., Poole M. S. Aesthetic design of e-commerce web pages – webpage complexity, order and preference. Electronic Commerce Research and Applications. 2012;11(4):420-40.

Ding D. X., Hu P. J. H., Sheng O. R. L. e-SELFQUAL: a scale for measuring online self-service quality. Journal of Business Research. 2011;64(1):508-15.

Embretson S. E. A multidimensional latent trait model for measuring learning and change. Psychometrika. 1991;56(3):495-515.

Embretson S., Reise S. P. Item response theory for psychologists.. 2000.

Éthier J., Hadaya P., Talbot J., Cadieux J. Interface design and emotions experienced on B2C Web sites: empirical testing of a research model. Computers in Human Behavior. 2008;24(2):2771-91.

Fang X., Holsapple C. W. An empirical study of web site navigation structures’ impacts on web site usability. Decision Support Systems. 2007;43(2):476-91.

Fassnacht M., Koese I. Quality of electronic services: conceptualizing and testing a hierarchical model. Journal of Service Research. 2006;9(1):19-37.

Flavián C., Guinalíu M. Consumer trust, perceived security and privacy policy: three basic elements of loyalty to a web site. Industrial Management & Data Systems. 2006;106(5):601-20.

Flavián C., Guinalíu M., Gurrea R. The influence of familiarity and usability on loyalty to online journalistic services: the role of user experience. Journal of Retailing and Consumer Services. 2006;13(5):363-75.

Fleck M. P. A., Bourdel M. C. Método de simulação e escolha de fatores na análise dos principais componentes. Revista de Saude Publica. 1998;32(3):267-72.

Ford J. K., Maccallum R. C., Tait M. The application of exploratory factor analysis in applied psychology: a critical review and analysis. Personnel Psychology. 1986;39(2):292-314.

Galletta D. F., Henry R., McCoy S., Polak P. Web site delays: how tolerant are users?. Journal of the Association for Information Systems. 2004;5(1):1-28.

Gata W., Gilang O. Analysis of information system quality of service on bsi academy’s environment using webqual methods, importance performance analysis and fishbone. Journal of Theoretical and Applied Information Technology. 2017;95(2):229.

Gefen D. Consumer loyalty in e-commerce. Journal of the Association for Information Systems. 2002;3(1):27-51.

Glas C. A. W. A rasch model with a multivariate distribution of ability.. Objective measurement: theory into practice. 1992:236-58.

Goldberg C. B., Allen D. G. Black and white and read all over: Race differences in reactions to recruitment Web sites. Human Resource Management. 2008;47(2):217-36.

González F. J. M., Palacios T. M. B. Quantitative evaluation of commercial websites: an empirical study of Spanish firms. International Journal of Information Management. 2004;24(4):313-28.

Gummerus J., Liljander V., Pura M., van Riel A. Customer loyalty to content-based web sites: the case of an online health-care service. Journal of Services Marketing. 2004;18(3):175-86.

Haga W. J., Zviran M. Information systems effectiveness: research designs for causal inference. Journal of Information Systems. 1994;4(2):141-66.

Hair J. F., Black W. C., Babin B. J., Anderson R. E., Tatham R. L. Análise multivariada de dados. 2009.

Hartig J., Höhler J. Representation of competencies in multidimensional IRT models with within-item and between-item multidimensionality. The Journal of Psychology. 2008;216(2):88-100.

Hasan B. Perceived irritation in online shopping: the impact of website design characteristics. Computers in Human Behavior. 2016;54:224-30.

Ibrahim E. E., Joseph M., Ibeh K. I. N. Customers’ perception of electronic service delivery in the UK retail banking sector. International Journal of Bank Marketing. 2006;24(7):475-93.

ISO9126: Information Technology — Software Product Evaluation — Quality Characteristics and Guidelines for Their Use. 1992.

Johnson R., Wichern D. Applied multivariate analysis. 2007.

Jun M., Yang Z., Kim D. S. Customers’ perceptions of online retailing service quality and their satisfaction. International Journal of Quality & Reliability Management. 2004;21(8):817-40.

Kaplan D., Krishnan R., Padman R., Peters J. KRISHNAN, R. PADMAN, R. PETERS, J. Assessing data quality in accounting information systems. Communications of the ACM. 1998;41(2):72-8.

Kettinger W. J., Lee C. C. Perceived service quality and user satisfaction with the information services function. Decision Sciences. 1994;25(5-6):737-66.

Kim C., Galliers R. D., Shin N., Ryoo J., Kim J. Factors influencing Internet shopping value and customer repurchase intention. Electronic Commerce Research and Applications. 2012;11(4):374-87.

Kim S., Stoel L. Dimensional hierarchy of retail website quality. Information & Management. 2004;41(5):619-33.

King W. R., Epstein B. J. Assessing information system value: an experimental study. Decision Sciences. 1983;14(1):34-45.

Kitchenham B., Pfleege S. L. Software quality: the elusive target. IEEE Software. 1996;13(1):12-21.

Kline P. The handbook of psychological testing. 2000.

Kulviwat S., Bruner 2nd. G. C., Kumar A., Nasco S. A., Clark T. Toward a unified theory of consumer acceptance of technology. Psychology and Marketing. 2007;24(12):1067-92.

Lindgaard G. Aesthetics, visual appeal, usability, and user satisfaction: What do the user’s eyes tell the user’s brain?. Australian Journal of Emerging Technologies and Society. 2007;5(1):1-14.

Lindgaard G., Gary J. F., Cathy D., Brown J. Attention Web Designers: You Have 50 Milliseconds to Make a Good First Impression!. Behaviour & Information Technology. 2006;25(2):115-26.

Lindroos K. Use quality and the World Wide Web. Information and Software Technology. 1997;39(12):827-36.

Loiacono E., Watson R., Goodhue D. WEBQUAL: a measure of web site quality. Marketing Educators’ Conference: Marketing Theory and Applications. 2002;13:432-7.

Lord F. M. A theory of test scores. 1952.

Lord F. M. Applications of item response theory to practical testing problems.. 1980.

McDonald R. P. The dimensionality of test and items. The British Journal of Mathematical and Statistical Psychology. 1981;33(1):161-83.

Nelson K. G. Global information systems quality: key issues and challenges. Journal of Global Information Management. 1996;4(4):4-14.

Nielsen J., Loranger H. Prioritizing web usability.. 2006.

Nielsen J., Tahir M. Homepage usability: 50 sites deconstructed.. 2002.

Nusair K., Kandampully J. The antecedents of customer satisfaction with online travel services: a conceptual model. European Business Review. 2008;20(1):4-19.

Oshima T. C., Miller M. D. Multidimensionality and item bias in item response theory. Applied Psychological Measurement. 1992;16(3):237-48.

Palmer J. W. Web site usability, design, and performance metrics. Information Systems Research. 2002;13(2):151-67.

Pandir M., Knight J. Homepage aesthetics: the search for preference factors and the challenges of subjectivity. Interacting with Computers. 2006;18(6):1351-70.

Parasuraman A., Zeithaml V. A., Malhotra A. E-S-Qual: a multiple-item scale for assessing electronic service quality. Journal of Service Research. 2005;7(3):213-33.

Pinterits A., Treiblmaier H., Pollach I. Environmental websites: an empirical investigation of functionality and accessibility. International Journal of Technology Policy and Management. 2006;6(1):103-19.

Podsakoff P. M., Ahearne M., Mackenzie S. B. Organizational citizenship behaviors and the quantity and quality of work group performance. The Journal of Applied Psychology. 1997;82(2):262-70.

R: a language and environment for statistical computing. 2012.

Rasch G. Probabilistic models for some intelligence and attainment tests.. 1960.

Rauch D., Hartig J. Multiple‐choice versus open‐ended response formats of reading test items: a two‐dimensional IRT analysis. Psychological Test and Assessment Modeling. 2010;52(4):354-79.

Reckase M. D. The difficulty of test items that measure more than one ability. Applied Psychological Measurement. 1985;9(4):401-12.

Reckase M. D. A linear logistic multidimensional model for dichotomous items response data.. Handbook of modern item response theory. 1997:271-86.

Reckase M. D. Multidimensional item response theory. 2009.

Reckase M. D., McKinley R. L. The discriminating power of items that measure more than one dimension. Applied Psychological Measurement. 1991;15(4):361-73.

Reise S. P., Widaman K. F., Pugh R. H. Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin. 1993;114(3):552-66.

Revelle W. Package ‘psych’: procedures for psychological, psychometric, and personality research. Version 1.2. 12. 2012.

Rust R. T. The rise of E-Service. Journal of Service Research. 2001;3(4):283-4.

Schmettow M., Vietze W. Introducing item response theory for measuring usability inspection processes. 2008.

Schneidewind N. F. Methodology for validating software metrics. IEEE Transactions on Software Engineering. 1992;18(5):410-22.

Seffah A., Donyaee M., Kline R. B., Padda H. K. Usability measurement and metrics: a consolidated model. Software Quality Control. 2006;14(2):159-78.

Serkan A., Eda A., Safak A. Re-assessment of E-S-Qual and E-RecS-Qual in a pure service setting. Journal of Business Research. 2010;63(3):232-40.

Silva R. G. S., Frio R. S., Sampaio C. H., Broilo P. L. Qualidade percebida do autosserviço online: validação da escala e-SELFQUAL no contexto brasileiro. Revista Brasileira de Marketing. 2015;14(2):138-54.

Singh T., Malik S., Sarkar D. E-commerce website quality assessment based on usability.. 2016:101-5.

Spector P. E., Van Katwyk P. T., Brannick M. T., Chen P. Y. When two factors don’t reflect two constructs: How item characteristics can produce artificial factors. Journal of Management. 1997;23(5):659-77.

Stepchenkova S., Tang L., Jang S. S., Kirilenko A. P., Morrison A. M. Benchmarking CVB website performance: Spatial and structural patterns. Tourism Management. 2010;31(5):611-20.

Stoyanov S. R., Hides L., Kavanagh D. J., Zelenko O., Tjondronegoro D., Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR mHealth and uHealth. 2015;3(1):e27.

Tavares H. R., Andrade D. F., Pereira C. A. B. Detection of determinant genes and diagnostic via item response theory. Genetics and Molecular Biology. 2004;27(4):679-85.

Taylor M. J., England D. Internet marketing: web site navigational design issues. Marketing Intelligence & Planning. 2006;24(1):77-85.

Tezza R., Bornia A. C., Andrade D. F. Measuring web usability using item response theory: principles, features and opportunities. Interacting with Computers. 2011;23(2):167-75.

Tezza R., Bornia A. C., Spenassato D., Trierweiller A. C. Empirical comparison of the multidimensional models of item response theory in e-commerce. Pesquisa Operacional. 2016;36(3):503-32.

van der Merwe R., Bekker J. A framework and methodology for evaluating e-commerce Web sites.. Internet Research Electronic Networking Applications and Policy. 2003;13(5):330-41.

Web content accessibility guidelines (WCA G) 2.0. W3C Recommendation. 2008.

Wang R., Storey V., Firth A. A framework for data quality research. IEEE Transactions on Knowledge and Data Engineering. 1995;7(4):623-40.

Xie M., Wang H., Goh T. N. Quality dimensions of Internet search engines. Journal of Information Science. 1998;24(5):365-72.

Yang Z., Cai S., Zhou Z., Zhou N. Development and validation of an instrument to measure user perceived service quality of information presenting Web portals. Information & Management. 2005;42(4):575-89.

Yang Z., Jun M., Peterson R. T. Measuring customer perceived online service quality: scale development and managerial implications. International Journal of Operations & Production Management. 2004;21(11):1149-74.

Zeithaml V. A., Parasuraman A., Malhotra A. E-service quality: definition, dimensions and conceptual model. 2000.

5defbc510e88252312b5f733 gp Articles
Links & Downloads

Gest. Prod.

Share this page
Page Sections