We combine multiple word representations based on semantic clusters extracted from the (Brown 1992) algorithm and syntactic clusters obtained from the Berkeley parser (Petrov et. al., 2006) in order to improve discriminative dependency parsing in the MSTParser framework (McDonald et. al., 2005). We also provide an ensemble method for combining diverse cluster-based models, which is a discriminative parsing analog to the generative product of experts model for parsing in (Petrov, 2010). The two contributions together significantly improves unlabeled dependency accuracy from 90.82\% to 92.13\%.