Sen. Rand Paul on Thursday took to the floor to prevent a vote on a two-year budget agreement, speaking indefinitely even if it meant forcing a shutdown.

"I think the country's worth a debate till 3 in the morning, frankly," the Kentucky Republican said shortly after his remarks began just before 6 p.m.

Paul has been urging Senate leaders to allow a vote on an amendment that would maintain the budget caps set by the 2011 Budget Control Act. Speaking earlier to reporters, Paul predicted he would be joined by 15 or 20 of his colleagues if his amendment were allowed to be brought up for a vote.

The budget deal reached by House and Senate leaders Wednesday would raise the sequester-imposed caps set by the 2011 law by $300 billion over the next two years. It also funds the government at current levels through March 23, allowing appropriators to write the details of the spending increases.

Senate leaders rejected Paul's request, arguing that the short time period before government funding is set to expire means they must deal with the budget agreement with some urgency and that it would not be fair to grant Paul his amendment without opening up the floor for amendments from other senators.

The clock will run out on debate at 1 a.m. Friday, but with unanimous consent from all members the Senate could move up the vote.

Shortly before 6 p.m. Thursday, Senate Majority Leader Mitch McConnell, R-Ky., and Minority Leader Chuck Schumer, D-N.Y., offered Paul the chance to offer a budget point of order, which would give him the opportunity to register his objections to the bill, but then the Senate could move forward to end debate.

Paul objected, complaining that the budget agreement increases spending without tackling underlying reasons that government deficits are increasing.

"When the Democrats are in power, Republicans appear to be the conservative party. But when Republicans are in power, it seems there is no conservative party," Paul said.

"The hypocrisy hangs in the air and chokes anyone with a sense of decency," he said.

Ultimately, the bill is likely to pass the Senate comfortably, and while it is facing more resistance in the House, it is also expected to be approved there – at whatever hour it arrives.

If a shutdown occurs, which now appears likely, it would be disruptive – even if it lasted just a few hours.

After Paul took the floor, the Office of Management and Budget sent notice to federal agencies that it "is currently preparing for a lapse in appropriations."