Article

Received 10.08.2025

Revised 24.11.2025

Accepted 29.12.2025

Retrieved from Vol. 28, No. 2, 2025

Pages 151 -167

Nesterenko, O. (2025). Methodological aspects and models for assessing the effectiveness of artificial intelligence in project management. The National Transport University Bulletin: A Scientific and Technical Journal, 28(2), 151-167. https://doi.org/10.32703/2617-9040-2025-46-11

Methodological aspects and models for assessing the effectiveness of artificial intelligence in project management

Oleksii Nesterenko

Rapid integration of artificial intelligence (AI) into project management offers significant potential to improve productivity through data automation, performance monitoring and schedule optimization. However, challenges such as “effective inefficiency” and the variability of AI model output complicate the assessment of its effectiveness. This article analyses the methodological aspects of evaluating AI effectiveness in project management, classifies existing methods (benchmarks, explainable AI, mutual information, psychometrics), identifies key challenges (biases, lack of standards, ethical constraints), and proposesnovel metrics-indicator of new competency activation (INCA), novelty coefficient in AI-Driven Project Management (NCAPM) and dynamic assessment of transition to new efficiency enabled by AI (DATNE) to measure innovation. The potential of these approachesfor transport infrastructure projects is indicated, where AI allows for the creation of fundamentally new opportunities in planning, service forecasting, and resource optimisation.Future directions include hybrid metrics and integration with decision support systems. The study underscores the need for interdisciplinary approaches to adapt AI evaluation to resource constrained project management environments

model; machine learning; benchmark; performance evaluation; methodology; cognitive models; system analysis; project management; artificial intelligence; SPPR
  1. Müller, R., Locatelli, G., Holzmann, V., Nilsson, M., & Sagay, T. (2024). Artificial intelligence and project management: Empirical overview, state of the art, and guidelines for future research. Project Management Journal, 55(1), 9–15. https://doi.org/10.1177/87569728231225198
  2. Mills, S., & Spencer, D. A. (2025). Efficient inefficiency: Organisational challenges of realising economic gains from AI. Journal of Business Research, 189, 115128. https://doi.org/10.1016/j.jbusres.2024.115128
  3. Edwards, J. (2025). How to measure AI efficiency and productivity gains. In InformationWeek (Ed.), AI and Machine Learning Insights. Available from: https://www.informationweek.com/machine-learning-ai/how-to-measure-ai-efficiency-and-productivity-gains
  4. Burden, J. (2024). Evaluating AI evaluation: Perils and prospects. arXiv preprint arXiv:2407.09221. https://doi.org/10.48550/arXiv.2407.09221
  5. Challapally, A., & Pease, C. (2025). AI trends and innovations in 2025. In Artificial Intelligence News (Ed.), Annual AI Report. Available from: https://www.artificialintelligence-news.com/wp-content/uploads/2025/08/aireport2025.pdf
  6. Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., … Bowman, S. (2019). SuperGLUE: A stickier benchmark for general-purpose language understanding systems. Advances in Neural Information Processing Systems, 32, 1–15. https://doi.org/10.48550/arXiv.1905.00537
  7. Zhang, M., Wang, H., Li, J., & Gao, H. (2020). Learned sketches for frequency estimation. Information Sciences, 507, 365–385. https://doi.org/10.1016/j.ins.2019.08.045
  8. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016, August). “Why should I trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135–1144). https://doi.org/10.1145/2939672.2939778
  9. Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., … Herrera, F. (2020). Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82–115. https://doi.org/10.1016/j.inffus.2019.12.012
  10. Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608. https://doi.org/10.48550/arXiv.1702.08608
  11. Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30. https://doi.org/10.48550/arXiv.1705.07874
  12. Olah, C., Mordvintsev, A., & Schubert, L. (2017). Feature visualization. Distill, 2(11), e7. https://distill.pub/2017/feature-visualization
  13. Tishby, N., Pereira, F. C., & Bialek, W. (2000). The information bottleneck method. arXiv preprint physics/0004057. https://doi.org/10.48550/arXiv.physics/0004057
  14. Hernandez-Orallo, J. (2017). Evaluating intelligence across species and machines. In The Measure of All Minds (pp. 50–100). Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/9781316596654
  15. McKinsey & Company. (2023). Generative AI’s impact on business in 2023. In The State of AI Report (pp. 10–25). New York, NY: McKinsey & Company. Available from: https://www.mckinsey.com/business-functions/mckinsey-digital/our-insights/the-state-of-ai-in-2023
  16. World Intellectual Property Organization (WIPO). (2023). Trends in AI technology development. Technology Trends 2023: Artificial Intelligence. Geneva, Switzerland: WIPO.
  17. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys, 54(6), 1–35. https://doi.org/10.1145/3457607
  18. MIT. (2025). Business applications of AI in 2025. The State of AI in Business 2025 (pp. 5–20). Cambridge, MA: MIT Press. Available from: https://www.mit.edu/ai-report-2025
  19. Nesterenko, O., & Kulbovskyi, I. (2024). Mathematical framework of transformer-based artificial intelligence architectures in large language models for the development of intelligent agents. Science and Technology Today, 5(46).

https://doi.org/10.32703/2617-9040-2025-46-11