Heavy-ion test data for 3D NAND flash memories is presented, along with a discussion of modern testing challenges and near-term plans for a broad survey of currently-available product lines

Campola, Michael

LaBel, Kenneth A.

Wilcox, Edward

NASA Technical Reports Server

COTS 3D NAND Flash: 
SEE Test Results and Challenges
Edward Wilcox, Michael Campola, Kenneth LaBel
NASA Goddard Space Flight Center
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
https://ntrs.nasa.gov/search.jsp?R=20180003142 2019-08-31T16:38:09+00:00Z
Outline
• Present State of Flash Memory
• NASA GSFC Testing Status
– Devices Under Test
– 3D NAND Flash Results To Date
• COTS Flash Memory Testing Challenges
– Packaging, Availability, and Electrical Access
• Future Plans
2
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Acronyms
• COTS: Commercial Off The Shelf
• ECC: Error-Correcting Code
• EDAC: Error Detection and 
Correction
• GEO: Geostationary Earth Orbit
• LET: Linear Energy Transfer
• MBU: Multiple Bit Upset
• MLC: Multi-level Cell
• NAND: Not AND (Flash 
Technology)
• NEPP: NASA Electronics and 
Packaging Program
• QLC: Quad-level Cell
• RBER: Raw Bit Error Rate
• SBU: Single Bit Upset
• SEE: Single Event Effects
• SEFI: Single Event Functional 
Interruption
• SEU: Single Event Upset
• SLC: Single-level Cell
• SSD: Solid State Drive
• TID: Total Ionizing Dose
• TLC: Triple-level Cell
• UBER: Uncorrected Bit Error Rate
3
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
State of Flash Memory
• Limitations of 2D Highly-Scaled Flash
• 3D Structures Maturing / Available
– Samsung 64-layer VNANDTM
– Toshiba / Western Digital / SanDisk 64-layer BiCS3TM
– Micron / Intel 64-layer
– Hynix 72-layer
• 1TB SSD <$500; 6Tb+ in a single package!
• Not just discrete components to worry about
– Integration into SoC- and SoB-type applications
4
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
3D NAND Structure
• Vertical flash strings, with 64 layers now common
• Not to be confused with 3D-stacking of multiple 
die in package
5
[https://www.3dincites.com/2014/08/samsungs-3d-vnand-
flash-product-spires-el-dorado/]
[http://www.micron.com]
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
NEPP / NASA GSFC Testing Status
• Previous NEPP SEE testing on Hynix 3D
– 36 layer vs new 72-layer
– D. Chen, NSREC 2017; TNS Jan. 2018
• 2017/2018 SEE testing on Micron MLC 3D NAND
– 32 Layer, floating gate technology
– 1Tb packages with four 256Gb die
– Limited availability / required teaming for procurement
– Re-used simple microcontroller test setup
• On-going SEE testing on variety of SSD modules
– Major manufacturers have their latest flash on SSDs
– Easy procurement BUT limited documentation
– No direct electrical access to memory devices
6
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Devices Under Test
• Micron MT29F1T08CMHBB
– 256Gb die; MLC; 32 layers; piece-part testing
• Micron MT29F768G08EEHBB
– 384Gb die; TLC; 32 layers; Crucial MX300 SSD module
• Intel
– 256Gb die; TLC; 64 layers; Intel 545 SSD module
• Samsung
– TLC; 64 layers; Samsung T5 Portable SSD
• SanDisk/Toshiba
– TLC; 64 layers; WD Blue 3D SSD module
– 15nm planar TLC; WD Blue SSD module
• Hynix H27QDG822C8R-BCG 
– Piece-part testing; MLC; 36 layers
7
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Micron MT29F1T08CMHBBJ4 
• Leveraged previous NASA test setups with 
Cortex-M4 microcontroller 
– Simple asynchronous interface
– Low-level electrical access; no mapping or abstraction
– No ECC We can actually see bit upsets…
8
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Micron MT29F1T08CMHBBJ4
9
In GEO, on the 
order of 800 
upsets per Tb 
per year…
But, MFG’s ECC 
requirement corrects 
on the order of 1% 
error rate!
So, SEU are lost in 
the noise? Not an 
issue?
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
3D NAND Angular Effects, Tilt vs. Roll 
• Dakai Chen on Hynix 36-layer MLC (TNS, 2018):
• Micron 32-layer MLC:
10
60° angle
of incidence
70° angle
increases 
MBU further
nBU: n upsets within 
a single word. 
Multiple SBU still 
possible from single 
particle strike
Chen, Wilcox, et al 
[TNS, 2018]
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
3D NAND Angular Effects, 
Constant LET
• How does “Cosine Law” apply with 3D NAND 
flash? 
11
Chen, Wilcox, et al 
[TNS, 2018]
Hynix 3D: Micron 3D:
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Data Pattern Dependence
• For Micron 3D NAND, no discernable pattern 
dependence (0’s and 1’s are being mapped 
evenly)
12
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Fluence Dependence
• Programmed-cell Vth is a distribution – not an ideal ON or 
OFF
• Consider some cells “easier” to upset than others 
• Reduced effect compared to previously observed Hynix 3D 
MLC flash.
• Relevant to understanding accelerated SEE test results!
13
Chen, Wilcox, et al [TNS, 2018]
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
TID Effects
• Let’s look at adding TID into the mix
• Shifts Vth distribution of flash cells… just like
– Heavy-ion particle strikes
– Program-Erase cycles
14
RBER =
Bit Errors/
Total Bits
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Micron Combined Effects
• How does TID before SEE affect error rate?
15
Look out as RBER > ECC… 
but unlikely circumstances
If we subtract the TID soft 
errors, heavy ion-induced  
SEU susceptibility appears 
unchanged.
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
SSD Test Setup
• Solid State Hard Drives are easy to buy, easy to 
use, and hard to test at the bit level!
– Abstraction, logical address mapping, EDAC, etc
• Number of upsets expected from SEU low
compared to memory size and built-in error rate
• Can we observe general trends from 
manufacturer-to-manufacturer in state-of-the-art 
3D NAND flash?
• Can TID or program/erase cycling magnify effect 
for easier comparison?
• Can we learn anything about effects of SEU on 
SSDs?
16To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devi.
SSD Test Results – WD Blue 3D SSD
• Irradiated to 1x106cm-2 N (LET 1.4 MeV·cm2/mg)
– Nothing observed on tester…
• Up to 1x108cm-2
– Still nothing
– Based on Micron 3D NAND testing we’d 
guess on the order of .0016 upsets/bit
– No reported uncorrectable errors
17To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devi.
Even when errors aren’t visible, 
other parametrics may suffer 
(SSD data rate lower by 50% 
after “hidden” SEUs)
WD Blue 3D Continued
18
• Pre-SEE testing: 10krad (Si) exposure
– No SSD errors noted following TID
• Irradiated to 1x107cm-2 Copper (LET 21.1 MeV·cm2/mg)
– Waited for full readback of drive… and nothing.
• Up to 1x108cm-2
– Based on Micron 3D NAND 
MLC testing we’d guess on 
the order of .010 upsets/bit.
– Errors abound (next slide)!
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
WD 3D Blue SSD Data
• Nothing abnormal noted immediately after run:
• But, after reading back drive:
19
S.M.A.R.T. attributes showed 
interesting data only after allowing 
drive controller to learn its own 
condition.
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
WD 3D Blue SSD Data
• Same LET, but at 65° angle
• Pre-SEE testing: 10krad (Si) exposure
• Irradiated to 1x108cm-2 Ar @ 65° (LET 21 MeV·cm2/mg)
– Several step irradiations with readbacks, no errors 
through 5x107cm-2.
– Big changes after final step:
20
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Other SSDs Tested
• Intel 64-layer TLC 
– 10 krad(Si) + 1x108cm-2 @ LET 1.4:
– All clean
– Separate device, 0 krad, 1x108cm-2 Copper (LET 21.1):
21
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Continued SSD Data
• Samsung 64-layer VNAND
• Micron 32-layer TLC
– 1x108cm-2 N (LET=1.4 MeVcm2/mg)
22
• Clean at 1x107cm-2 Copper 
(LET 21.1 MeVcm2/mg)
• Few errors at 5x107cm-2.
• Stopped mounting for ~1 hour
• Fully erasable and now normal
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Challenges
• SSD testing adds layers of abstraction and 
mapping on top of error-correcting code.
• Effectively impossible to see any individual 
errors, even MBUs and minor SEFIs; major SEFI 
events likely to dominate error-response
• Must test as a black-box system
– Ok if you’re trying to characterize a black-box system, but 
limited insight into marginal degradation at part level
23
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA
Future Plans
• Generational scaling in 3D layer count and 
feature size will continue
– Test piece parts when able, but SSD-type testing 
possible as well
• Evaluate combined effects, particularly as 
TLC/QLC cells continue to erode margins and 
increase RBER
– TID/SEE/Endurance/Retention all tightly coupled
24
To be presented by Edward Wilcox at the Single Event Effects (SEE) Symposium and Military and Aerospace Programmable Logic Devices (MAPLD) Workshop, 
May 21-24, 2018, in La Jolla, CA


COTS 3D NAND Flash: SEE Test Results and Challenges

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server