Loading...
|
Please use this identifier to cite or link to this item:
https://nccur.lib.nccu.edu.tw/handle/140.119/32641
|
| Title: | 電腦輔助克漏詞多選題出題系統之研究
A Study on Computer Aided Generation of Multiple-Choice Cloze Items |
| Authors: | 王俊弘
Wang , Chun-Hung |
| Contributors: | 劉昭麟
Liu , Chao-Lin
王俊弘
Wang , Chun-Hung |
| Keywords: | 電腦輔助語言學習
自動產生試題
試題編寫工具
詞義辨析
自然語言處理
Computer-Assisted Language Learning
Automatic Item Generation
Authoring Tools
Word Sense Disambiguation
Natural Language Processing |
| Date: | 2003 |
| Issue Date: | 2009-09-17 13:54:53 (UTC+8) |
| Abstract: | 多選題測驗試題已證明能有效地評估學生的學習成效，然而，以人為方式建立題庫是一件耗時費力的工作。藉由電腦高速運算的能力，電腦輔助產生試題系統能有效率地建置大規模的題庫，同時減少人為的干預而得以保持試題的隱密性。受惠於網路上充裕的文字資源，本研究發展一套克漏詞試題出題系統，利用既有的語料自動產生涵蓋各種不同主題的克漏詞試題。藉由分析歷屆大學入學考試的資料，系統可產生類似難度的模擬試題，並且得到出題人員在遴選測驗標的方面的規律性。在產生試題的過程中導入詞義辨析的演算法，利用詞典與selectional preference模型的輔助，分析句子中特定詞彙的語義，以擷取包含測驗編撰者所要測驗的詞義的句子，並以collocation為基礎的方法篩選誘答選項。實驗結果顯示系統可在每產生1.6道試題中，得到1道可用的試題。我們嘗試產生不同類型的試題，並將這套系統融入網路線上英文測驗的環境中，依學生的作答情形分析試題的鑑別度。
Multiple-choice tests have proved to be an efficient tool for measuring students’ achievement. Manually constructing tests items, however, is a time- consuming and labor-intensive task. Harnessing the computing power of computers, computer-assisted item generation offers the possibility of creating large amount of items, thereby alleviating the problem of keeping the items secure. With the abundant text resource on the Web, this study develops a system capable of generating cloze items that cover a wide range of topics based on existing corpra. By analyzing training data from the College Entrance Examinations in Taiwan, we identify special regularities of the test items, and our system can generate items of similar style based on results of the analysis. We propose a word sense disambiguation-based method for locating sentences in which designated words carry specific senses, and apply collocation-based methods for selecting distractors. Experimental results indicate that our system was able to produce a usable item for every 1.6 items it returned. We try to create different types of items and integrate the reported item generator in a Web-based system for learning English. The outcome of on-line examinations is analyzed in order to estimate the item discrimination of the test items generated by our system. |
| Reference: | [1] 郭生玉，心理與教育測驗，台北：精華書局，1989。
[2] 鄭恆雄，陳凌霞，宋麗梅等，八十七年度英文命題實驗計畫研究報告，台北：大學入學考試中心，1999。
[3] Berger, A. L., Della Pietra, S. A. & Della Pietra, V. J. (1996). A maximum entropy approach to natural language processing. Computational Linguistics, 22 (1), 39–71.
[4] Coniam, D. (1997). A preliminary inquiry into using corpus word frequency data in the automatic generation of English language cloze tests. Computer Assisted Language Instruction Consortium, 16 (2–4), 15–33.
[5] Deane, P. & Sheehan, K. (2003). Automatic item generation via frame semantics. Education Testing Service: http://www.ets.org/research/dload/ncme03-deane.pdf .
[6] Dennis, I., Handley, S., Bradon, P., Evans, J. & Nestead, S. (2002). Approaches to modeling item-generative tests. In: Item Generation for Test Development [12], 53–72.
[7] Dorr, B. J., Levow, G.-A. & Lin, D. (2000). Large-scale construction of a Chinese- English semantic hierarchy. Technical report, University of Maryland, College Park, MD.
[8] Fillmore, C. J. & Atkins, B.T. (1992). Towards a frame based lexicon: the semantics of RISK and its neighbors. Frames, Fields and Constraints (Lawrence Erlbaum Associates), 75–102.
[9] Florian, R. & Wicentowski, R. (2002). Unsupervised Italian word sense disambiguation using WordNets and unlabeled corpora. Proceedings of the SIGLEX/SENSEVAL Workshop on Word Sense Disambiguation: Recent Success and Future Directions, 67–73.
[10] Gao, Z.-M. & Liu, C.-L. (2004). A Web-based assessment and profiling system for college English. Proceedings of the Eleventh International Conference on Computer Assisted Instruction, CD–ROM.
[11] Huang, S.-M., Liu, C.-L. & Gao, Z.-M. (2003). Toward computer assisted learning for English dictation. Proceedings of the 2003 Joint Conference on Artificial Intelligence, Fuzzy Systems, and Grey Systems, CD–ROM.
[12] Irvine, S. H. & Kyllonen, P. C. (2002). Item Generation for Test Development (Lawrence Erlbaum Associates).
[13] Johns, T. http://web.bham.ac.uk/johnstf/timcall.htm.
[14] Lesk, M. (1986). Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. Proceedings of the Special Group for Design of Communications Conference, 24–26.
[15] Lin, D. (1998). Dependency-based evaluation of MINIPAR. Proceedings of the Workshop on the Evaluation of Parsing Systems in the First International Conference on Language Resources and Evaluation.
[16] Liu, C.-L. (2004). Using mutual information for adaptive student assessments. Proceedings of the Fourth IEEE International Conference on Advanced Learning Technologies, to appear.
[17] Madsen, H. (1983). Techniques in Testing (Oxford University Press).
[18] Manning, C. D. & Schütze, H. (1999). Foundations of statistical natural language processing (MIT Press).
[19] Marcus, M. P., Santorini, B. & Marcinkiewicz, M. A. (1993). Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19 (2), 313–330.
[20] Martin, B. & Mitrovic A. (2002). Automatic problem generation in constraint-based tutors. Proceedings of the Conference on Intelligent Tutoring System, 388–398.
[21] Miller, G. A. (1995). WordNet: a lexical database for English. Communications of the ACM, 38 (11), 39–41.
[22] Mitkov, R. & Ha, L. A. (2003). Computer-aided generation of multiple-choice tests. Proceedings of the HLT-NAACL 2003 Workshop on Building Educational Applications Using Natural Language Processing, 17–22.
[23] Ohlsson, S. (1991). Constraint-based student modeling. Proceedings of the NATO Advanced Research Workshop on Student Modeling, 167–189.
[24] Oranje, A. (2003). Automatic item generation applied to the national assessment of educational progress: Exploring a multilevel structural equation model for categorized variables. Education Testing Service: http://www.ets.org/research/dload/ncme03-andreas.pdf.
[25] Poel, C. J. & Weatherly, S. D. (1997). A cloze look at placement testing. Shiken: JALT (Japanese Assoc. for Language Teaching) Testing & Evaluation SIG Newsletter, 1 (1), 4–10.
[26] Ratnaparkhi, A. (1996). A maximum entropy part-of-speech tagger. Proceedings of the Conference on Empirical Methods in Natural Language Processing, 133–142.
[27] Resnik, P. (1997). Selectional preference and sense disambiguation. Proceedings of the Applied Natural Language Processing Workshop on Tagging Text with Lexical Semantics: Why, What and How, 52–57.
[28] Reynar, J. C. & Ratnaparkhi, A. (1997). A maximum entropy approach to identifying sentence boundaries. Proceedings of the Conference on Applied Natural Language Processing, 16–19.
[29] Sheehan, K. M., Deane, P. & Kostin, I. (2003). A partially automated system for generating passage-based multiple-choice verbal reasoning items. Paper presented at the National Council on Measurement in Education Annual Meeting.
[30] Steven, V. (1991). Classroom concordancing: vocabulary materials derived from relevant authentic text. English for Specific Purposes, 10 (1), 35–46.
[31] Wang, C.-H., Liu, C.-L. & Gao, Z.-M. (2003). Toward computer assisted item generation for English vocabulary tests. Proceedings of the 2003 Joint Conference on Artificial Intelligence, Fuzzy Systems, and Grey Systems, CD–ROM.
[32] Wilks, Y. & Stevenson, M. (1997). Combining independent knowledge sources for word sense disambiguation. Proceedings of the Conference on Recent Advances in Natural Language Processing, 1–7.
[33] Wilson, E. (1997). The automatic generation of CALL exercises from general corpora. Proceedings of the Conference on Teaching and Language corpora, 116–130.
[34] Yang, X.-F. & Li, T.-Q. (2002). A study of semantic disambiguation based on HowNet. Computational Linguistics and Chinese Language Processing, 7 (1), 47–78.
[35] Yarowsky, D. (1992). Word-sense disambiguation using statistical models of Roget`s categories trained on large corpora. Proceedings of COLING-92, 454–460. |
| Description: | 國立政治大學
資訊科學學系
91753024
92 |
| Source URI: | http://thesis.lib.nccu.edu.tw/record/#G0091753024 |
| Data Type: | thesis |
| Appears in Collections: | [資訊科學系] 學位論文
|
All items in 政大典藏 are protected by copyright, with all rights reserved.
|
