With the emergence of artificial intelligence, personality rights—which had not yet been granted independent status under the purview of intellectual property rights—face more serious threats to their preservation. As AI has developed, creating procedures that protect people's rights has presented a number of difficulties. Advancements in technology such as Deep Fake enable individuals to perpetrate identity theft, which can result in several offenses under both the IT Act and the criminal laws that are in place in most other nations. The truth is that a large number of manufacturers, almost all ISPs, and online service providers use proprietary software and artificial intelligence systems in one way or another to mine, gather, analyze, and arrange user-generated content that is hosted on their own web platforms, on servers run by private third parties, or on platforms used by their advertising partners. How can people legally control and/or profit from the commercial use of their own works that are protected by intellectual property rights, as well as their own name, image, voice, and other personally identifying characteristics, in user-generated content created by them and used by private companies using proprietary machine learning systems, also known as artificial intelligence software? It is commonly known that AI, with enough data and the right training, may produce effortlessly effortless near-perfect clones of a celebrity's persona; an almost ideal replacement for the persona of an author; and content that is hard for customers to distinguish from endorsements or official appearances. The present paper critically examines the protection of personality rights under the umbrella of privacy and publicity rights, highlighting the challenges posed by the growth of technology, especially with the artificial intelligence at the stage of general and super artificial intelligent machines. It further delves into the criminal acts being committed using the artificially intelligent systems that defies the concept of privacy and personality resulting from the lack of legislative framework for its governance.