Wang, FakaiX-ray images have been widely used by radiologists for disease diagnosis. For Rheumatoid Arthritis (RA), Joint Space Narrowing (JSN) is one major symptom that can be read from X-ray images. In this thesis, we investigate the JSN classification for RA diagnosis in terms of methodology, data analysis, neural network models, performance analysis. First, we perform the statistical analysis of X-ray data and design a baseline convolutional neural network (CNN). We show algorithms to extract joint patches. Then we conduct prediction analysis. Second, we design the fusion model to harness the correlation between the same type of joints. Sharing information within one X-ray image would increase the prediction performance. We also compare unified classifiers and separate classifiers. Third, we design the attention map model for joints with complex contexts, which filters out unrelated surroundings. We conclude that our models give good JSN prediction for Rheumatoid Arthritis.enThesisComputer engineeringattention modelconvolutional neural networkfusion modeljoint space narrowingRheumatoid Arthritis