Ejected Electrons

I realized that my previous results on the electrons being ejected from the simulation didn’t make any sense. The electric fields from the positive ions should pull the electrons back and it should take a relatively long time for the electrons to leave the simulation. We can estimate this by computing the RMS velocity of the electrons which is $7.26 \times 10^5\sqrt{T_\text{eV}}$ m/s from our simulation set up. The corners of the target are around 7 $\mu$m away from the edge of the simulation at a minimum.

Taking this into account, we find that the minimum time for an electron to leave the simulation would be 9.66ps at 1eV or 305fs at 1000eV. If we additionally add the protons in to slow the electons down, this time would get much larger. As a result, we are not losing many electrons on the time-scale of less than 1ps when there is no laser present. The below plot confirms this.

Using the exec and eval functions

To execute dynamically generated code in python, the exec and eval functions are very useful. Below would be an example of how one might use the functions

foo1 = 1
foo2 = 2
idx = random.randint(0, 1)
exec('var = foo{}'.format(idx))
var = eval('foo{}'.format(idx))

The exec function evaluates a whole line of code (including assignment to a variable) and the eval function just evaluates an expression and returns an output (which would need to be assigned). I have found these functions useful in extracting data from EPOCH due to the way in which the data needs to be extracted, but I have ran into issues using exec. It seems that sometimes, the variable assigned in exec is not properly assigned so that when I try to use it later, I get an error. As a result, I think I should just stick to using eval from now on.

Heating Results

I attempted to run many 2D simulations without a laser for a longer time frame (2ps) but these were too long and expensive and I only ran some fraction of the total amount I wanted to run. I wanted to see if I could make any conclusions from the simulations without the laser.

The above plot shows a clear inverse relation to the number of electron PPC as expected. Additionally, we can see that the amount of heating does increase with the initial temperature. In these plots, I am comparing the temperature increase to that of a simulation with a high number of PPC (because all of them have the electron temperature dropping over time). The plot above is from 300-500 or 800fs and the plot below is from 500-1000 or 2000fs. I didn’t start at 0fs, because I wanted to give the ions and electrons some time to stabilize after their initialization.

Additionally, we can see that the Arber Formula over-predicts the simulation heating (in Ricky’s simulations with the laser, it did the opposite: under-predicting). In my simulations I did last year, I found this had to do with the size of the target: a smaller target heats up less than one that fills up the whole simulation volume. Due to the difference in the no laser vs laser simulations, it seems like it would be difficult to predict the amount of heating with the laser on from data from simulations without a laser.

E Field Fluctuations

I tried to look into the fluctuations in the electric field and see how that could be related to the temperature.

This picture shows the E-field Fluctuations and Temperature and how they evolve in time. The E-field fluctuations jump up and down and are not very easy to read. A better metric might be a plot of the average fluctuation and average electron temperature normalized to the fluctuation/electron temp at a given initial temperature. Below is that plot

Here, we can see that the electric field fluctuations and electron temperature both go up linearly with the initial temperature of the electrons in the simulations. The average electron temperature is lower than the y=x line because the electrons are losing energy to the ions over time in the simulation.

One problem with measuring the field fluctuations is that the simulation predicts a much higher value than the Hockney formula

\begin{equation} \langle E^2 \rangle = \frac{k_B T}{4 \pi \epsilon_0 \lambda_D^2} \text{Log}(1 + u_{max^2}) \label{eq:e2} \end{equation}

The Hockney formula gives around $10^8 V^2/m^2$ and the simulation gives around $10^{18} V^2/m^2$ which is many orders of magnitude off.

Number Density

Here is a plot of the electron number density evolving with time

it doesn’t seem to change with time. Since we don’t have a laser, I don’t see why the Electric Fields should fluctuate so much if nothing is moving.

HPLSE Paper Submission

We received news that our manuscript has been rejected by HPLSE. They gave some useful feedback, here are some of the main takeaways.

1. We don’t have a complex enough dataset to justify using machine learning. We only had 3 inputs (effectively two, because all the focal distance does is change the effective intensity on target). Additionally, we limited ourself to 20,000 training data points or less which is not enough to justify using neural networks
2. Didn’t go into detail of the hyperparameter tuning and selection and use of validation/holdout set. This is something I can fix and improve upon
3. I fit three different single task models instead of one multi-task model for the regressions. This is also something I should be able to fix.
4. Use of a synthetic dataset is not that useful without extending to experiment or simulation. It is not clear that this synthetic dataset even approximates the true behavior that we might find in experiment.

We had a Zoom Meeting and talked about next steps with this paper. We decided that it may be useful to focus in on just the regression algorithms and add in the pre-pulse model effects as an extra parameter to make the paper more interesting. However, doing this may make Jack’s future paper seem to similar, so I’m not sure if trying to get two papers out of this project is a reasonable goal, especially since one of the reviewers said in its current form, I do not feel transfer to any other journal is suitable.

Notes on Machine Learning

I decided to look through the textbook that Jack used for his Big Data Machine Learning course: Hands On Machine Learning with Scikit Learn and see if I can take any lesson from it. Here, I’m going to summarize what I’ve learned

1. Overfitting is when the model is too complex relative to amount and noisiness of data. Solutions would include simplifying the model architecture, regularization, gathering more training data, and reducing noise (or removing outliers). We can reduce overfitting by early stopping: stopping training once the validation error reaches a minimum
2. Underfitting is when the model is too simple. To fix this, we should try a different, more complex model and find better features.
3. scikit-learn allows custom transformers to be defined, we should use this so that we can get everything to work within the sklearn framework instead of making our own transformers
4. MinMaxScaler() and StandardScaler() are the two most common ways to apply feature scaling
5. Using Pipelines is useful to chain different things together
6. Use K-fold cross validation with regression models to ensure we are not overfitting to training data
7. Can use pickle or joblib to save models
8. For hyperparameter tuning, we can use RandomizedSearchCV followed by GridSearchCV. Alternatively, if our parameter space is large, we can instead use BayesSearchCV from skopt for a better (non-random) searching of the hyper-parameter space
9. Ridge and Lasso Regressions are good alternatives to Linear Regression that add some regularization
10. Activation functions introduce non-linearity to the neural networks: ReLU, LeakyReLU, tanh, logistic, etc. LeakyReLU is a good default. ELU is better but may take more time to compute. tanh and logistic seem to be obsolete.
11. Loss Functions: MSE, MAE, Huber
12. Use the same number of neurons per layer in a Neural Network. Start with just 1-2 hidden layers and add more layers until you start to overfit
13. Adam (and Nadam) optimizers work much better than the default SGD. We can still using Learning rate scheduling to decay learning rate as we continue training.
14. Stick with a relatively small batch size of around 32.
15. Use Batch Normalization if training and inference time is not much of an issue.
16. Transfer learning does not work well with small dense networks (like what we are using for the Fuchs dataset)

Things to Do

• Work on DPP Poster Next Week and upload and update my presenter profile
• Modify ML paper and train models on OSC using new skorch framework.
• Continue working on W3 Tutorials for Python and Javascript in my free time.