This article presents a study of the application of hierarchical classifiers based on the hierarchical mixtures of experts. In particular we present an extension of our work that explores the use of linear classifiers and a hybrid model that combines back propagation neural networks with linear classifiers. We test this model using the UMLS as the classification structure and a subset of medical abstracts from MEDLINE. Our results confirm that using the hierarchical structure of the classification vocabulary improves categorization performance.