Li'aan (oath of condemnation) is derived from Li'aan (curse) because each one of the spouses curses himself/herself (in the fifth time) if he/she is lying. The origin of it is in his (swt)'s saying:

“And for those who accuse their wives but have no witnesses except themselves, let the testimony of one of them be four testimonies (i.e. testifies four times) by Allah that He is one of those who speak the truth. And the fifth testimony should be the invoking of the Curse of Allah on Himself if He is of those who tell a lie against her. But it shall avert the punishment (of stoning to death) from her, if she bears witness four times by Allah, that He (her husband) is telling a lie. And the fifth (testimony) should be that the Wrath of Allah be upon her if He (her husband) speaks the truth” [An- Nur: 6-9]

Abu Dawood reports through his isnad (line of transmitters) on the authority of Ibn Abbas (ra) who said: “Hilal ibn Umayyah was one of three people whose repentance was accepted by Allah. One night He returned home from his land and found a man with his wife. He witnessed with his eyes and heard with his ears. He did not stir Him till the morning. Next day He went to the Messenger of Allah  in the morning, and said: “’O Messenger of Allah! (pbuh) I came to my wife at night and found a man with her. I saw with my own eyes and heard with my own ears’. The Prophet of Allah (pbuh) disliked what He described and He took it seriously. There upon the following Qur’anic verse came down: “And for those who accuse their wives, but have no witnesses except themselves, let the testimony of one of them...” Then the Messenger of Allah (pbuh) was relieved and said: ‘Glad tidings for you, Hilal. Allah, the Exalted, has made an easy way out for you’. Hilal said: ‘I expected that from my Lord’. The Messenger of Allah (pbuh) said: ‘Send for her’. She then came. The Messenger of Allah (pbuh) recited (the verses) to them that the punishment in the next world was more severe than that in this world. Hilal said:’I swear by Allah, I spoke the truth against her’. She said: ‘He told a lie’. The Messenger of Allah (pbuh) said: ‘Apply the method of invoking curses between them’. Hilal was told: ‘Bear witness’. So He bore witness before Allah four times that He spoke the truth.When He was about to utter a fifth time, He was told:’Hilal, fear Allah, for the punishment in this world is easier than that in the next world; and this is the deciding one that will surely cause punishment to you.’ He said: ‘I swear by Allah. Allah will not punish me for this act, as He did not cause me to be flogged for this act’. So He bore witness a fifth time invoking the curse of Allah on Him if He was one of those who told lies. Then she was told: ‘Testify’. So she gave testimony before Allah that He was a liar. When she was going to testify a fifth time, she was told: ‘Fear Allah, for the punishment in this world is easier than that in the next world. This is the deciding one that will surely cause punishment to you’. She hesitated for a moment, and then said: ‘By Allah, I shall not disgrace my people’. So she testified a fifth time invoking the curse of Allah on her if He spoke the truth. The Messenger of Allah (pbuh) separated them from each other and decided that there would be no dwelling or maintenance for her from the husband.”

So if a man accused his wife and said to her: “you have committed fornication or you are a fornicator or I have seen you commit fornication”, and He did not bring proof, then the punishment for defamation will be imposed on Him if they do not invoke curse upon each other. If He invoked curses upon her and she does not do the same to Him, then the punishment will be imposed on her due to the saying of Allah (swt):

“But it shall avert the punishment from her, if she bears witness four times by Allah” [An- Nur: 8] The punishment that will be averted from her is the punishment that is mentioned in his (swt)’s saying:

“And let a party of the believers witness their punishment” [An- Nur: 2]

Because when Hilal b. Umayya accused his wife of fornication and came to the Prophet (pbuh) He told Him to send for her and He made them invoke curses upon each other. This is one of the specific cases in which fornication is proven, i.e. the case in which the husband accuses his wife of fornication and proves it by invoking curses upon her and she does not. If she invoked curses upon Him then it is not proven. Her refusal to invoke curses proves fornication and obliges the imposition of the punishment on her due to his Li’aan.

If they both invoked curses upon each other the judge will separate them (terminate the marriage) and they can never marry each other again. She is forbidden for Him forever because the Prophet (pbuh) separated the two cursing parties. Malik reports from Nafi from Ibn Umar that a man accused and made Li'aan to his wife in the time of the Messenger of Allah and rejected her child so the Messenger separated them and granted the child to the woman. Sahl b. Sad reported that the Sunnah which continued with respect to the two cursing parties is that they are separated and shall never come back together again. This separation by imprecation (Li’aan) constitutes an annulment (Faskh) of the marriage contract because this type of separation requires prohibition of marriage forever. It is not allowed for Him even if He declares Himself to have lied. However if He returns and admits to lying then she has the right for the punishment to be imposed on Him. And the child’s paternity will be ascribed to Him whether He admitted lying before or after the imprecation.

The imprecation which absolves the husband from the punishment and obliges it to be imposed on the wife if she refuses to invoke curses, is that the husband should say in the presence of the judge; ‘I bear witness by Allah that she has committed fornication’ and He points to her. If she is not present He mentions her name and lineage until He repeats it four times and then at the fifth time He stops. It will be said to Him fear Allah this is the deciding one, for the punishment in this world is easier than that in the next world. If He refuses to stop He should say; (in the fifth) ‘The curse of Allah is on Him if He was one of those who told lies in regards to his accusation that she committed fornication’. And she should say: ‘I bear witness that He has lied’ (four times) and stops at the fifth. She will be told to fear Allah as the man was told. If she refuses to stop she should say (in the fifth): ‘Allah’s wrath be on her if He was telling the truth’ with respect to accusing her of fornication. If they have a child then the child is mentioned in the Li’aan. When He said: ‘I bear witness she has committed fornication’ He should say: ‘and this child is not mine’. And she should say: ‘By Allah He has lied, and this child is his’.

This is the method of mutual imprecation (Li'aan) and these are its words and sentences. That is why if the woman gives birth to a child and He says this child is not from me or this is not my child, there is no punishment on Him because this is not defamation (Qadhf). However He will be asked: If He meant she has committed adultery and she gave birth from Zina, this is defamation (Qadhf) and it is proven through imprecation. If He said “I meant that He does not physically resemble me or I doubt she had intercourse and the boy is from the one who had intercourse” or things similar to that. In such case there will be no punishment on Him, and the child’s lineage will be ascribed to Him because He has not defamed (Qadhf) her. There is no imprecation in such situations because one of the conditions of imprecation is defamation (Qadhf).