Combinatorial scientific computing plays an important enabling role in computational science, particularly in high performance scientific computing. In this thesis, we will describe our work on optimizing matrix-vector multiplication using combinatorial techniques. Our research has focused on two different problems in combinatorial scientific computing, both involving matrix-vector multiplication, and both are solved using hypergraph models. For both of these problems, the cost of the combinatorial optimization process can be effectively amortized over many matrix-vector products. The first problem we address is optimization of serial matrix-vector multiplication for relatively small, dense matrices that arise in finite element assembly. Previous work showed that combinatorial optimization of matrix-vector multiplication can lead to faster assembly of finite element stiffness matrices by eliminating redundant operations. Based on a graph model characterizing row relationships, a more efficient set of operations can be generated to perform matrix-vector multiplication. We improved this graph model by extending the set of binary row relationships and using hypergraphs to model more complicated row relationships, yielding significantly improved results over previous models. The second problem we address is parallel matrix-vector multiplication for large sparse matrices. Parallel sparse matrix-vector multiplication is a particularly important numerical kernel in computational science. We have focused on optimizing the parallel performance of this operation by reducing the communication volume through smarter, two-dimensional matrix partitioning. We have developed and implemented a recursive algorithm based on nested dissection to partition structurally symmetric matrices. In general, this method has proven to be the best available for partitioning structurally symmetric matrices (when considering both volume and partitioning time) and has shown great promise for information retrieval matrices. We also developed a second, simpler method that is fast and works well for many symmetric matrices. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]