Uniform sampling of binary matrices with fixed margins is known as a difficult problem. Two classes of algorithms to sample from a distribution not too different from the uniform are studied in the literature: importance sampling and Markov chain Monte Carlo (MCMC). Existing MCMC algorithms converge slowly, require a long burn-in period and yield highly dependent samples. Chen et al. developed an importance sampling algorithm that is highly efficient for relatively small tables. For larger but still moderate sized tables (300x30) Chen et al.'s algorithm is less efficient. This article develops a new MCMC algorithm that converges much faster than the existing ones and that is more efficient than Chen's algorithm for large problems. Its stationary distribution is uniform. The algorithm is extended to the case of square matrices with fixed diagonal for applications in social network theory.