You are able to keep one lockpick in your jail cell, which can be used to try to escape from prison. If you cannot pay your bounty, you will be sent to the nearest town jail. If you wish to persuade the guard again, wait 2 days and the option will be available again. If your speech is 75 or higher, this option will always be present (given that the guard has not been persuaded already). If successful, you will raise Speech, but will not clear your bounty. Success is dependent on the Speech skill. You can attempt to persuade the guard that you are not worth the effort of dealing with, due to your low bounty, and should be allowed to go free. Occasionally another option will appear if you have a low bounty. If you had some previous bounty as well, this won't get cleared. Note, the bounty cleared is only for the crime you were recently caught doing. The Bribe Guards Speech perk also grants you this option. To do this however, you have to do five jobs in the region and complete the special task. The amount of gold required to bribe a guard is usually much larger than the bounty for small crimes, but less than one for murder and doing so will raise the Speech skill, immediately clear your bounty and avoid having items confiscated. Thieves Guild members have the option to bribe the guard. This additional option will allow you to keep stolen items.

Thieves Guild members have an additional option to pay a much lower fine, if they so choose. If you pay your bounty, you will be immediately transported outside the nearest town jail, and all stolen items in your possession will be seized. This only applies to bounties less than 3,000 gold. This will immediately clear your bounty, but can only be done once per hold. If you are the Thane of a hold, you can declare this to the guard. There are a few ways you can remove the bounty over your head

Skyrim guard wait i know you