While looking for another quote, I came across these paragraphs (one from the conclusion of his memoirs, the other from an early 1861 letter). Grant is not an impartial source, but I like when people agree with me.
The cause of the great War of the Rebellion against the
will have to be attributed to slavery. For some years before the war began it was a trite saying among some politicians that “A state half slave and half free cannot exist.” All must become slave or all free, or the state will go down. I took no part myself in any such view of the case at the time, but since the war is over, reviewing the whole question, I have come to the conclusion that the saying is quite true. United States
Slavery was an institution that required unusual guarantees for its security wherever it existed; and in a country like ours where the larger portion of it was free territory inhabited by an intelligent and well-to-do population, the people would naturally have but little sympathy with demands upon them for its protection. Hence the people of the South were dependent upon keeping control of the general government to secure the perpetuation of their favorite institution. They were enabled to maintain this control long after the States where slavery existed had ceased to have the controlling power, through the assistance they received from odd men here and there throughout the Northern states. They saw their power waning, and this led them to encroach upon the prerogatives and independence of the Northern States by enacting such laws as the Fugitive Slave Law. By this law every Northern man was obliged, when properly summoned, to turn out and help apprehend the runaway slave of a Southern man. Northern marshals become slave-catchers, and Northern courts had to contribute to the support and protection of the institution.
This was a degradation which the North would not permit any longer than until they could get the power to expunge such law from the statute books. Prior to the time of these encroachments the great majority of the people of the North had no particular quarrel with slavery, so long as they were not forced to have it themselves. But they were not willing to play the role of police for the South in the protection of this particular institution.
No impartial man can conceal from himself the fact that in all these troubles the South have been the aggressors and the Administration has stood purely on the defensive, more on the defensive than she would dared to have done but for her consciousness of strength and the certainty of right prevailing in the end….In all this I can but see the doom of Slavery. The North do not want nor will they want, to interfere with the institution. But they will refuse for all time to give it protection unless the South shall return soon to their allegiance.…Letter to Frederick Dent,
19 April 1861
From Ulysses S. Grant Memoirs and Selected Letters (published by Library of Congress in 1990)