This dissertation introduces a new multimedia security system for the performance of object recognition and multimedia encryption in security and medical applications. The system embeds an enhancement and multimedia encryption process into the traditional recognition system in order to improve the efficiency and accuracy of object detection and recognition while protecting multimedia data. The dissertation then focuses on the system's enhancement and encryption processes, and presents contributions to the area of image enhancement and multimedia encryption. For the purposes of image enhancement, a new enhancement measure called the second-derivative-like measure of enhancement (SDME) is introduced to quantitatively evaluate the enhancement algorithm's performance. This is followed by the introduction of a new 3D CT baggage image enhancement algorithm for homeland security applications. Computer simulations and comparisons demonstrate that the presented algorithm significantly improves the visual quality of objects in the original CT images while reducing background noise. The quantitative SDME measure results and 3D visualizations verify the algorithm's excellent enhancement performance. To improve the visual quality of medical images and thereby improve the efficiency and accuracy of early cancer detection, a new nonlinear filter, called the Alpha-Weighted Quadratic Filter (AWQF), is integrated with human visual system based decomposition and unsharp masking techniques to enhance mammograms for breast cancer detection. To enhance prostate MR images for prostate cancer detection, the same filter is integrated with alpha-trimmed mean separation and logarithmic enhancement techniques. Simulation results and comparisons are given to demonstrate the enhancement algorithms' excellent performance when it comes to enhancing mammograms and prostate MR images. In order to enhance the efficiency and security of existing encryption algorithms, five parametric recursive sequences and their transforms are developed and applied to multimedia scrambling/encryption. The recursive sequence transforms are integrated with the Fibonacci bit-plane decomposition method and the newly introduced (" n, k, p")-Gray code bit-plane decomposition for image and object encryption. Based on the concept of using one set of security keys to encrypt the original image and a different set of security keys to reconstruct the image to obtain the final encrypted image, a discrete parametric cosine transform is used for image encryption. Despite the fact that the edge map has been used traditionally for image compression, enhancement, and recognition, it has never been used for image encryption. This dissertation investigates the use of the edge map for applications in image encryption. The edge map is integrated with the 3D Cat Map for image encryption and with the chaotic logistic map for encrypting medical images for the sake of privacy protection. The concept of the edge map is then extended further to produce a binary "key-image", which can either be a bit-plane or an edge map obtained from any other image. Computer simulations, comparisons and security analysis demonstrate the excellent performance of the presented encryption algorithms when it comes to multimedia encryption. The contributions of this dissertation are as follows: (1) A new multimedia security system for object detection and multimedia encryption (2) Image enhancement: (a) A new SDME measure for quantitatively evaluating the enhancement algorithm's performance (b) A new 3D CT baggage image enhancement algorithm using alpha weighted mean separation for homeland security applications (c) A new alpha weighted quadratic filter for mammogram enhancement (d) A new human visual system decomposition based algorithm for mammogram enhancement (e) A new nonlinear unsharp masking scheme for mammogram enhancement (f) A new prostate MR image enhancement algorithm using the alpha-trimmed mean separation and nonlinear filtering (g) A new logarithmic enhancement algorithm using nonlinear filtering for prostate MR image enhancement (3) Multimedia encryption: (a) Five new parametric recursive sequences and their transforms for multimedia encryption, including the truncated P-Fibonacci sequence, P-Lucas sequence, ("n", "k," "p")--Gray code, P-recursive sequence and the parametric M-sequence. (b) A new 2D P-recursive transform (c) Two P-recursive transform based multimedia encryption algorithms in the spatial and frequency domains (d) A new truncated Fibonacci p-code bit-plane decomposition (e) A new ("n," "k," "p")--Gray code bit-plane decomposition (f) A new image encryption algorithm using the P-Fibonacci transform and decomposition (g) A new selective object encryption algorithm using the truncated Fibonacci p-code bit-plane decomposition (h) A new image encryption algorithm using the ("n," "k," "p")--Gray code and its decomposition (i) A new image encryption algorithm using the Discrete Parametric Cosine Transform (j) A new image encryption algorithm using the edge map and a new 3D Cat Map (k) A new medical image encryption algorithm using the edge map and chaotic logistic map (l) Two image encryption algorithms using binary key-images: one uses bit-plane and the other uses the edge map. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]