Apple has developed a key chip in-house.

In the previous article "Apple's Self-developed Image Sensor? A Stunning Patent Unveiled", we introduced that Apple has newly released a patent named "Stacked Pixel Image Sensor with High Dynamic Range and Low Noise".

The latest news recently indicates that Apple is laying the foundation for a mobile phone camera capable of instantly freezing an entire scene. This is the global shutter. A brand - new patent confirms this. It means clearer motion pictures, fewer flickering bands, and more professional video effects. Here is a simple version that anyone can easily understand.

Imagine a referee shouting "Stop". Everyone stops simultaneously. Then the camera records everything it sees. Since everyone stops at the same time, straight lines remain straight. The effects of LED walls and stage lights are better. It's easier to capture a game with the "flash". Here are some principles:

1. A rolling shutter reads an image line by line. Fast - moving objects can make straight lines bend and cause jitter.

2. A global shutter captures each pixel at the same moment and then calmly reads it.

3. Apple's patent shows that there is a tiny "waiting room" inside each pixel, so that the mobile phone can freeze the entire frame at once. This is a great idea. Based on this application, you can explore the following solutions:

Next, we will explain Apple's patent in simple English.

First, we imagine each pixel as a three - layer stack. The first floor is the light - collecting area, the second floor is a small waiting room, and the third floor is a micro - meter that converts light into digital signals.

The specific working principle is as follows: During exposure, the first layer will be filled with light. At the freezing moment, each pixel will simultaneously move the light from the first layer to the second layer. In this way, the entire frame is locked. Then, the mobile phone moves the stored light from the second layer to the third layer line by line to generate a photo. The middle layer is the secret. It provides each pixel with a dedicated "stopping point" to store the frozen image.

So, what's different about Apple's approach in this solution?

As shown in the figure, Apple's method is to build upwards rather than horizontally. Looking at many current designs on the market, they place the "waiting room" next to the spotlight. This takes up space and light. Apple stacks the channels vertically, thus saving more light - collecting space.

Among them, one part undertakes two tasks - the middle layer stores the image and helps move the image, which makes the pixels compact and easy for the mobile phone to operate. There is less stray light in the frozen frame. The patent describes the shields and trenches that protect the "waiting room" when the shutter is "closed". A clearer frozen frame means a clearer image. As shown below, the drawings include back - illuminated lighting for autofocus and split pixels. These are tools for dense smartphone sensors, not bulky laboratory parts.

Apple's Self - description of this Patent

As Apple said. An image sensor may include multiple pixels, and each pixel may include a photodiode having a charge accumulation region ("PD"), a floating diffusion region ("FD"), and a charge transfer region vertically located between the PD and the FD. The vertical charge transfer region may include a first charge modulation region ("P1"), a second charge modulation region ("P2"), and a third charge modulation region ("P3"). The image sensor may operate in a global shutter mode, where P2 may be used as an in - pixel charge storage region for temporarily storing charge during the transfer of charge from the PD to the FD via P1, P2, and P3.

As for this patent, it generally relates to an image sensor, and more specifically to an image sensor pixel with vertically integrated multiphase charge transfer technology for image capture in a global shutter mode.

Apple points out that image capture devices (e.g., cameras) are widely used in various electronic devices, such as mobile devices (e.g., smartphones, tablets, laptops, etc.), robotic devices, or security monitoring devices. An image capture device may include an image sensor with multiple light - collecting pixels. Each pixel may include a photodiode. The image capture device can capture light from the environment and transmit the light to the image sensor. When exposed to light, the photodiode of a pixel can accumulate charge. At readout, one or more transistors can be used to read the charge of the photodiode from the photodiode to generate an analog image signal. The analog image signal can be converted into a digital signal and further processed to generate an image.

As Apple said, in some embodiments, an image sensor may include multiple light - collecting pixels, for example, these pixels are organized into a pixel array having one or more rows of pixels and one or more columns of pixels. In some embodiments, the image sensor may be a CMOS (Complementary Metal - Oxide - Semiconductor) image sensor, a CCD (Charge - Coupled Device) image sensor, etc. In some embodiments, the image sensor may be part of an image capture device (e.g., a camera), and the image capture device may also be part of an electronic device (e.g., a mobile device (e.g., a smartphone, a tablet, a laptop, etc.), a robotic device, or a security monitoring device). In some embodiments, each pixel of the image sensor may include at least one photodiode, which includes a charge accumulation region (hereinafter referred to as "PD"), a floating diffusion region (hereinafter referred to as "FD"), and a charge transfer region vertically located between the PD and the FD.

"When each pixel is exposed to light, the PD can accumulate charge or photocarriers. At readout, at least part of the charge can be transferred from the PD to the FD, thereby generating an analog image signal (e.g., an analog voltage) on the FD, which can be further accessed through a pixel output line outside the pixel. In certain embodiments, the analog image signal accessed through the pixel output line can be further processed, for example, using an analog - to - digital converter for analog - to - digital conversion, and then digitally processed by an image signal processor (ISP) to generate one or more images." Apple emphasizes.

Apple further points out that generally, a given image acquisition device can operate in a rolling shutter mode or a global shutter mode. In the rolling shutter mode, when the "wave" of readout sweeps across the image sensor, different rows of the pixel array of the image sensor of the image acquisition device can be exposed at different times. For example, the pixels of the pixel array can be exposed, and the image signals of the pixels can be read out sequentially, for example, line by line from the top to the bottom of the pixel array. For example, pixels in the same row can be read out simultaneously, while pixels in the same column but different rows can be read out sequentially one by one. Therefore, in the rolling shutter mode, the image sensor can record an image sequentially line by line rather than capturing the entire image at once.

In contrast, in the global shutter mode, the exposure time of all pixels is the same, which means that each pixel in the image sensor can start and end exposure simultaneously. Therefore, the entire image can be recorded at once. Since different "lines" of the image are recorded at different times, the rolling shutter can cause color and/or tone changes in the captured image. In certain applications, such as high - speed photography or recording, this can cause serious interference and greatly affect the quality of the captured image.

Therefore, in certain embodiments, the global shutter may be preferred. However, in certain embodiments, even if the pixels of the image sensor end exposure simultaneously, their image signals may still be read out sequentially like a rolling shutter, for example, line by line. Therefore, the image sensor may need "memory" to temporarily store (a) the charge of the pixels (e.g., in the charge domain) and/or (b) the analog or digital image signals (e.g., in the voltage domain) at the end of exposure until a single pixel is read.

To solve this problem, in some embodiments, each pixel of the image sensor disclosed herein can include an in - pixel charge storage area. At the end of exposure, the charge can be transferred from the pixel detector (PD) to the in - pixel charge storage area. The charge can be temporarily stored there until the pixel is read out. At readout, the charge can be transferred from the in - pixel charge storage area to the fluorescence detector (FD), and then the analog image information can be further accessed from the fluorescence detector (FD) through the pixel output line.

"Those skilled in the art should understand that the image sensor disclosed herein can provide multiple advantages." Apple said.

First, it can provide an in - pixel charge storage area for each pixel to temporarily store the pixel's charge, thus enabling the image sensor to operate in the global shutter mode. In addition, the "memory" is integrated as part of the pixel to store the charge inside the pixel, thereby eliminating or at least reducing other additional storage components (e.g., a storage chip on the image sensor, etc.). Therefore, this can reduce the number of components of the image sensor, reduce the space occupied by the image sensor, and/or increase the pixel density of the sensor.

Apple emphasizes. The methods described herein can be implemented in the form of software, hardware, or a combination thereof in different embodiments. In addition, the order of the blocks of the method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc. Those skilled in the art can make various modifications and changes after benefiting from the present disclosure. The various embodiments described herein are for illustration only, not for limitation. Various changes, modifications, additions, and improvements can be made. Therefore, multiple instances can be provided for the components described herein, except for a single instance. The boundaries between various components, operations, and data storage are somewhat arbitrary, and specific operations will be described in the context of specific illustrative configurations. Other functional allocations can be envisaged, and these allocations may fall within the scope of the subsequent claims.

Finally, the structures and functions presented in the form of discrete components in the example configuration can be implemented as combined structures or components. These and other changes, modifications, additions, and improvements may fall within the scope of the embodiments defined by the subsequent claims.

This article is from the WeChat official account "Semiconductor Industry Observation" (ID: icbank), author: Editorial Department. It is published by 36Kr with authorization.