diff --git a/docs/_sources/tccs-13.rst.txt b/docs/_sources/tccs-13.rst.txt index 56b16be..bd4d331 100644 --- a/docs/_sources/tccs-13.rst.txt +++ b/docs/_sources/tccs-13.rst.txt @@ -4,7 +4,7 @@ TCCS-13 Here we describe the steps to reproduce the results in: -Landa-Marbán, D., Sandve, T.H., and Gasda, S.E. 2025. A Coarsening Approach to the Troll Aquifer Model. Submitted to SINTEF Proceedings. https://doi.org/10.13140/RG.2.2.24886.41283 +* Landa-Marbán, D., Sandve, T.H., and Gasda, S.E. 2025. A Coarsening Approach to the Troll Aquifer Model. Submitted to SINTEF Proceedings. https://doi.org/10.13140/RG.2.2.24886.41283 To this end, the deck `ORIGINAL.DATA `_ has the same dimensions and number of total cells (12,450,809) as the Troll aquifer model, and the rest of the properties are @@ -23,6 +23,13 @@ For the first figures, `plopm `_ and `p pip install git+https://github.com/cssr-tools/plopm.git pip install git+https://github.com/cssr-tools/pycopm.git +To run the optimization, `Everest `_ is needed, +which is installed via `ert `_ when `pycopm `_ is installed with the command above. + +.. note:: + + Python 3.14 was not supported for ert at the time this was written, then you need Python 3.12 or 3.13. + The following commands generate Figures 1 and 2 (a few features such as the transmissibilities, wells, and sensor are added using PowerPoint) using `FIG1.DATA `_: @@ -36,7 +43,7 @@ The following commands generate Figures 1 and 2 (a few features such as the tran plopm -i FIG1 -v poro -c '#bfebf2' -z 0 -grid 'black,1e0' -y '[5,-1]' -ylnum 5 -xlnum 7 -r 0 -remove 0,0,1,1 -d 24,16 -f 60 -save fig1 plopm -i 'COARSENED_PERMS FIG1 COARSENED_TRANS' -v 'pressure - 0pressure' -subfigs 3,1 -r 1 -z 0 -delax 1 -y '[5,-1]' -cformat .0f -grid 'black,1e0' -cbsfax 0.1,0.95,0.8,0.02 -suptitle 0 -clabel 'Pressure increase [bar]' -cnum 5 -t 'Coarsened (permeabilities) Before coarsening Coarsened (transmissibilities)' -d 24,48 -f 80 -save fig2 -For Figs. 3b, 5, 6, and 8, 9, and 10: +For Figs. 3b, 5, 6, and 8, 9, and 10 using ORIGINAL.DATA in the `expreccs/tccs-13/original `_ folder: .. code-block:: bash @@ -53,21 +60,13 @@ For Figs. 3b, 5, 6, and 8, 9, and 10: plopm -i 'ORIGINAL' -v 'imbnum' -s ',,1:217' -r 0 -translate '[-495000,-6.605e6]' -x '[0,95000]' -xunits km -xlnum 2 -y '[0,160000]' -yunits km -yformat .0f -ylnum 2 -xformat .0f -remove 0,0,1,1 -c '203;203;203' -save fig10a plopm -i 'ORIGINAL COARSENED' -v 'overpres' -s ',,1:217 ,,1:5' -translate '[-495000,-6.605e6]' -x '[0,95000]' -xunits km -xlnum 2 -y '[0,160000]' -yunits km -yformat .0f -ylnum 2 -xformat .0f -subfigs 1,2 -d 16,12 -cbsfax 0.1,0.95,0.8,0.02 -f 24 -cnum 5 -suptitle 0 -clabel 'Overpressure (p - p$_{lim}$) [bar], t=25 years' -delax 1 -remove 1,0,0,0 -t 'Troll aquifer model Coarsened version' -b '[-126.7,-1.1]' -c cet_diverging_rainbow_bgymr_45_85_c67 -save fig10bc -Figs. 4 and 7 are generated using `ResInsight `_. To run the optimization, `Everest `_ is needed. Since it has been merged to -`ERT `_, it seems there are some issues using it via ERT. Then, the walk around is to use the version before merging. One way to get this installed is to -create a new virtual environment (to avoid version conflicts, tested with Python 3.13) and execute: - -.. code-block:: bash - - pip install ert==11.0.8 - pip install git+https://github.com/equinor/everest.git - pip install mako - -Once installed, then you could adapt the coarsened generated files in the `coarsened folder `_ to run the optimization: +Figs. 4 and 7 are generated using `ResInsight `_. + +Using the coarsened files from pycopm, you could adapt the exisiting files in the `coarsened folder `_ to run the optimization: .. code-block:: bash - everest run coarsened.yml + everest run coarsened.yml --skip-prompt .. note:: @@ -79,7 +78,7 @@ After the study, to generate Fig. 11, execute the `postprocessing.py file

TCCS-13

Here we describe the steps to reproduce the results in:

-

Landa-Marbán, D., Sandve, T.H., and Gasda, S.E. 2025. A Coarsening Approach to the Troll Aquifer Model. Submitted to SINTEF Proceedings. https://doi.org/10.13140/RG.2.2.24886.41283

+

To this end, the deck ORIGINAL.DATA has the same dimensions and number of total cells (12,450,809) as the Troll aquifer model, and the rest of the properties are set to common homogeneous values from literature, while the well locations match the ones described in the TCCS-13 paper. @@ -99,6 +101,12 @@

TCCS-13 pip install git+https://github.com/cssr-tools/pycopm.git +

To run the optimization, Everest is needed, +which is installed via ert when pycopm is installed with the command above.

+
+

Note

+

Python 3.14 was not supported for ert at the time this was written, then you need Python 3.12 or 3.13.

+

The following commands generate Figures 1 and 2 (a few features such as the transmissibilities, wells, and sensor are added using PowerPoint) using FIG1.DATA:

pycopm -i FIG1.DATA -z 1:4 -m all -a max -w COARSENED_TRANS -l TRANS -t 2
@@ -110,7 +118,7 @@ 
TCCS-13
 plopm -i 'COARSENED_PERMS FIG1 COARSENED_TRANS' -v 'pressure - 0pressure' -subfigs 3,1 -r 1 -z 0 -delax 1 -y '[5,-1]' -cformat .0f -grid 'black,1e0' -cbsfax 0.1,0.95,0.8,0.02 -suptitle 0 -clabel 'Pressure increase [bar]' -cnum 5 -t 'Coarsened (permeabilities)  Before coarsening  Coarsened (transmissibilities)' -d 24,48 -f 80 -save fig2
-

For Figs. 3b, 5, 6, and 8, 9, and 10:

+

For Figs. 3b, 5, 6, and 8, 9, and 10 using ORIGINAL.DATA in the expreccs/tccs-13/original folder:

flow ORIGINAL.DATA --enable-opm-rst-file=true --linear-solver=cpr_trueimpes --time-step-control=newtoniterationcount --newton-min-iterations=1 --tolerance-cnv-relaxed=1e-2 --relaxed-max-pv-fraction=0
 pycopm -i ORIGINAL.DATA -t 2 -a max -z 1:30,31:56,57:111,112:116,117:217 -w COARSENED -l C
 flow COARSENED.DATA --enable-opm-rst-file=true --linear-solver=cpr_trueimpes --time-step-control=newtoniterationcount --newton-min-iterations=1 --tolerance-cnv-relaxed=1e-2 --relaxed-max-pv-fraction=0
@@ -125,16 +133,9 @@ 
TCCS-13
 plopm -i 'ORIGINAL COARSENED' -v 'overpres' -s ',,1:217 ,,1:5' -translate '[-495000,-6.605e6]' -x '[0,95000]' -xunits km -xlnum 2 -y '[0,160000]' -yunits km -yformat .0f -ylnum 2 -xformat .0f -subfigs 1,2 -d 16,12 -cbsfax 0.1,0.95,0.8,0.02 -f 24 -cnum 5 -suptitle 0 -clabel 'Overpressure (p - p$_{lim}$) [bar], t=25 years' -delax 1 -remove 1,0,0,0 -t 'Troll aquifer model  Coarsened version' -b '[-126.7,-1.1]' -c cet_diverging_rainbow_bgymr_45_85_c67 -save fig10bc
-

Figs. 4 and 7 are generated using ResInsight. To run the optimization, Everest is needed. Since it has been merged to -ERT, it seems there are some issues using it via ERT. Then, the walk around is to use the version before merging. One way to get this installed is to -create a new virtual environment (to avoid version conflicts, tested with Python 3.13) and execute:

-
pip install ert==11.0.8
-pip install git+https://github.com/equinor/everest.git
-pip install mako
-
-
-

Once installed, then you could adapt the coarsened generated files in the coarsened folder to run the optimization:

-
everest run coarsened.yml
+
Figs. 4 and 7 are generated using ResInsight.

+
Using the coarsened files from pycopm, you could adapt the exisiting files in the coarsened folder to run the optimization:

+
everest run coarsened.yml --skip-prompt
 

 

 

@@ -145,7 +146,7 @@ 
TCCS-13
 
python3 postprocessing.py
 

 

-
Finally, you can look at the improved well locations in the generated figures/optimal_solution folder, and take those locations in the ORGININAL.DATA deck
+
Finally, you can look at the improved well locations in the generated figures/optimal_solution folder, and take those locations into the ORGININAL.DATA deck
 and save it as IMPROVED.DATA, similar to the coarsened version to COARSENED_IMPROVED.DATA, to generate Fig. 12:

 
flow IMPROVED.DATA --enable-opm-rst-file=true --linear-solver=cpr_trueimpes --time-step-control=newtoniterationcount --newton-min-iterations=1 --tolerance-cnv-relaxed=1e-2 --relaxed-max-pv-fraction=0
 flow COARSENED_IMPROVED.DATA --enable-opm-rst-file=true --linear-solver=cpr_trueimpes --time-step-control=newtoniterationcount --newton-min-iterations=1 --tolerance-cnv-relaxed=1e-2 --relaxed-max-pv-fraction=0
diff --git a/docs/text/tccs-13.rst b/docs/text/tccs-13.rst
index 56b16be..bd4d331 100644
--- a/docs/text/tccs-13.rst
+++ b/docs/text/tccs-13.rst
@@ -4,7 +4,7 @@ TCCS-13
 
 Here we describe the steps to reproduce the results in:
 
-Landa-Marbán, D., Sandve, T.H., and Gasda, S.E. 2025. A Coarsening Approach to the Troll Aquifer Model. Submitted to SINTEF Proceedings. https://doi.org/10.13140/RG.2.2.24886.41283
+* Landa-Marbán, D., Sandve, T.H., and Gasda, S.E. 2025. A Coarsening Approach to the Troll Aquifer Model. Submitted to SINTEF Proceedings. https://doi.org/10.13140/RG.2.2.24886.41283
 
 To this end, the deck `ORIGINAL.DATA `_ has the 
 same dimensions and number of total cells (12,450,809) as the Troll aquifer model, and the rest of the properties are 
@@ -23,6 +23,13 @@ For the first figures, `plopm `_ and `p
     pip install git+https://github.com/cssr-tools/plopm.git
     pip install git+https://github.com/cssr-tools/pycopm.git
 
+To run the optimization, `Everest `_ is needed,
+which is installed via `ert `_ when `pycopm `_ is installed with the command above.
+
+.. note::
+
+    Python 3.14 was not supported for ert at the time this was written, then you need Python 3.12 or 3.13.
+
 The following commands generate Figures 1 and 2 (a few features such as the transmissibilities, wells, and sensor are added using PowerPoint) using 
 `FIG1.DATA `_:
 
@@ -36,7 +43,7 @@ The following commands generate Figures 1 and 2 (a few features such as the tran
     plopm -i FIG1 -v poro -c '#bfebf2' -z 0 -grid 'black,1e0' -y '[5,-1]' -ylnum 5 -xlnum 7 -r 0 -remove 0,0,1,1 -d 24,16 -f 60 -save fig1
     plopm -i 'COARSENED_PERMS FIG1 COARSENED_TRANS' -v 'pressure - 0pressure' -subfigs 3,1 -r 1 -z 0 -delax 1 -y '[5,-1]' -cformat .0f -grid 'black,1e0' -cbsfax 0.1,0.95,0.8,0.02 -suptitle 0 -clabel 'Pressure increase [bar]' -cnum 5 -t 'Coarsened (permeabilities)  Before coarsening  Coarsened (transmissibilities)' -d 24,48 -f 80 -save fig2
 
-For Figs. 3b, 5, 6, and 8, 9, and 10:
+For Figs. 3b, 5, 6, and 8, 9, and 10 using ORIGINAL.DATA in the `expreccs/tccs-13/original `_ folder:
 
 .. code-block:: bash
 
@@ -53,21 +60,13 @@ For Figs. 3b, 5, 6, and 8, 9, and 10:
     plopm -i 'ORIGINAL' -v 'imbnum' -s ',,1:217' -r 0 -translate '[-495000,-6.605e6]' -x '[0,95000]' -xunits km -xlnum 2 -y '[0,160000]' -yunits km -yformat .0f -ylnum 2 -xformat .0f -remove 0,0,1,1 -c '203;203;203' -save fig10a
     plopm -i 'ORIGINAL COARSENED' -v 'overpres' -s ',,1:217 ,,1:5' -translate '[-495000,-6.605e6]' -x '[0,95000]' -xunits km -xlnum 2 -y '[0,160000]' -yunits km -yformat .0f -ylnum 2 -xformat .0f -subfigs 1,2 -d 16,12 -cbsfax 0.1,0.95,0.8,0.02 -f 24 -cnum 5 -suptitle 0 -clabel 'Overpressure (p - p$_{lim}$) [bar], t=25 years' -delax 1 -remove 1,0,0,0 -t 'Troll aquifer model  Coarsened version' -b '[-126.7,-1.1]' -c cet_diverging_rainbow_bgymr_45_85_c67 -save fig10bc
 
-Figs. 4 and 7 are generated using `ResInsight `_. To run the optimization, `Everest `_ is needed. Since it has been merged to 
-`ERT `_, it seems there are some issues using it via ERT. Then, the walk around is to use the version before merging. One way to get this installed is to
-create a new virtual environment (to avoid version conflicts, tested with Python 3.13) and execute:
-
-.. code-block:: bash
-
-    pip install ert==11.0.8
-    pip install git+https://github.com/equinor/everest.git
-    pip install mako
-
-Once installed, then you could adapt the coarsened generated files in the `coarsened folder `_ to run the optimization:
+Figs. 4 and 7 are generated using `ResInsight `_. 
+ 
+Using the coarsened files from pycopm, you could adapt the exisiting files in the `coarsened folder `_ to run the optimization:
 
 .. code-block:: bash
 
-    everest run coarsened.yml
+    everest run coarsened.yml --skip-prompt
 
 .. note::
 
@@ -79,7 +78,7 @@ After the study, to generate Fig. 11, execute the `postprocessing.py file  config["mthr"]:
         with open("func", "w", encoding="utf-8") as f:
-            f.write("-1e10")
+            f.write("-1e2")
         return
 
     with open("func", "w", encoding="utf-8") as f:
diff --git a/tccs-13/postprocessing.py b/tccs-13/postprocessing.py
index 72076f3..bf3fc36 100644
--- a/tccs-13/postprocessing.py
+++ b/tccs-13/postprocessing.py
@@ -8,7 +8,6 @@
 
 import argparse
 import json
-import csv
 import os
 import numpy as np
 import matplotlib
@@ -35,7 +34,7 @@
 def postprocessing():
     """Main function to postprocess everest results (differential_evolution)"""
     cmdargs = load_parser()
-    dic = {"folder": f"{os.getcwd()}/{cmdargs['folder'].strip()}"}
+    dic = {"folder": os.path.abspath(cmdargs['folder'].strip())}
     with open(f"{dic['folder']}/configuration.json", "r", encoding="utf-8") as f:
         dic = {**json.load(f), **dic}
 
@@ -43,80 +42,68 @@ def postprocessing():
     if not os.path.exists(f"{dic['folder']}/figures"):
         os.system(f"mkdir {dic['folder']}/figures")
 
+    dic["ind_batch"], dic["ind_sim"] = [0, 0], [0, 0]
+    for i in range(4):
+        dic[f"s{i}"] = []
+        dic[f"x{i}"] = 0
+    process_optimization_results(dic)
     plot_optimization_results(dic)
     plt.rcParams.update({"axes.grid": False})
     plot_optimization_details(dic)
     find_optimal(dic)
 
 
+def process_optimization_results(dic):
+    """Process the optimization results over steps (batches)"""
+    os.chdir(dic["folder"])
+    dic["tot_eval"], dic["optimal"], dic["optimization"], where = 0, [], [], []
+    improved = -np.inf
+    root = "everest_output/sim_output"
+    for n in range(len(os.listdir(root))):
+        for m in range(len(os.listdir(f"{root}/batch_{n}/realization_0"))):
+            file = root + f"/batch_{n}/realization_0/evaluation_{m}/func"
+            if os.path.exists(file):
+                value = np.genfromtxt(file)
+                dic["tot_eval"] += 1
+                if value == -1e3:
+                    dic["x0"] += 1
+                elif value == -1e2:
+                    dic["x1"] += 1
+                elif value < 0:
+                    dic["x2"] += 1
+                else:
+                    dic["x3"] += 1
+                if value > -10:
+                    improved = max(improved, value)
+                    dic["optimal"].append(value)
+                    where.append([n, m])
+        for i in range(4):
+            dic[f"s{i}"].append(dic[f"x{i}"])
+            dic[f"x{i}"] = 0
+        dic["optimization"].append(improved)
+    ind = int(np.nanargmax(np.array(dic["optimal"])))
+    dic["optimal"] = dic["optimal"][ind]
+    dic["ind_batch"][1] = where[ind][0]
+    dic["ind_sim"][1] = where[ind][1]
+
+
 def find_optimal(dic):
     """Find the well locations and folder for the 'optimal' obtained solution"""
-    file = f"{dic['folder']}/everest_output/optimization_output/optimizer_output.txt"
-    with open(file, "r", encoding="utf8") as file:
-        optimal = -np.float16((file.readlines()[-1].strip()).split()[4])
-    file = f"{dic['folder']}/everest_output/optimization_output/results.txt"
-    dic["ind_batch"] = [0, 0]
-    dic["ind_sim"] = [0, 0]
-    batch_size = 0
-    with open(file, "r", encoding="utf8") as file:
-        for row in csv.reader(file):
-            if len(row) > 0:
-                if ((row[0].strip()).split()[0]).isdigit():
-                    if int((row[0].strip()).split()[1]) == 0:
-                        batch_size += 1
-                    if np.float16((row[0].strip()).split()[2]) == optimal:
-                        dic["ind_batch"][1] = int(
-                            int((row[0].strip()).split()[0]) / batch_size
-                        )
-                        dic["ind_sim"][1] = (
-                            int((row[0].strip()).split()[0])
-                            - dic["ind_batch"][1] * batch_size
-                        )
-                        break
+    
     for i, name in enumerate(["initial_guess", "optimal_solution"]):
         if not os.path.exists(f"{dic['folder']}/figures/{name}"):
             os.system(f"mkdir {dic['folder']}/figures/{name}")
         os.chdir(f"{dic['folder']}/figures/{name}")
         os.system(
             f"cp {dic['folder']}/everest_output/sim_output/"
-            + f"batch_{dic['ind_batch'][i]}/geo_realization_0/simulation_"
+            + f"batch_{dic['ind_batch'][i]}/realization_0/evaluation_"
             + f"{dic['ind_sim'][i]}/point_0.json ."
         )
         os.system(f"python3 {dic['folder']}/jobs/locations.py --mode post")
 
 
-def read_results(dic):
-    """Get the values over the optimization steps"""
-    file = f"{dic['folder']}/everest_output/optimization_output/results.txt"
-    n, dic["tot_eval"] = 0, 0
-    with open(file, "r", encoding="utf8") as file:
-        for row in csv.reader(file):
-            if len(row) > 0:
-                if ((row[0].strip()).split()[0]).isdigit():
-                    dic["tot_eval"] += 1
-                    if n != int((row[0].strip()).split()[1]):
-                        for i in range(4):
-                            dic[f"s{i}"].append(dic[f"x{i}"])
-                            dic[f"x{i}"] = 0
-                        n += 1
-                    if float((row[0].strip()).split()[2]) == -1e20:
-                        dic["x0"] += 1
-                    elif float((row[0].strip()).split()[2]) == -1e10:
-                        dic["x1"] += 1
-                    elif float((row[0].strip()).split()[2]) < 0:
-                        dic["x2"] += 1
-                    else:
-                        dic["x3"] += 1
-    for i in range(4):
-        dic[f"s{i}"].append(dic[f"x{i}"])
-
-
 def plot_optimization_details(dic):
     """Plot the number of failed and succeed simulations over steps"""
-    for i in range(4):
-        dic[f"s{i}"] = []
-        dic[f"x{i}"] = 0
-    read_results(dic)
     colors = ["k", "m", "r", "g", "b"]
     names = [
         f"Failed (well on inactive region, no={sum(dic['s0'])})",
@@ -152,31 +139,22 @@ def plot_optimization_details(dic):
 
 def plot_optimization_results(dic):
     """Plot the optimization values over steps"""
-    optimization = []
-    file = f"{dic['folder']}/everest_output/optimization_output/optimizer_output.txt"
-    with open(file, "r", encoding="utf8") as file:
-        for j, row in enumerate(csv.reader(file)):
-            optimization.append(
-                [j + 1, -float((row[0].strip()).split()[4])]
-            )
     fig, axis = plt.subplots()
     axis.step(
-        [step[0] for step in optimization],
-        [value[1] for value in optimization],
+        range(1, len(dic["optimization"]) + 1),
+        dic["optimization"],
         lw=5,
         color="b",
     )
     axis.set_title("Optimization results")
     axis.set_ylabel("min[(p$_{lim}$-p)/p$_{lim}$] [-]")
     axis.set_xlabel(r"Step [\#]")
-    print(optimization)
-    exit()
-    if optimization[-1][0] + 1 < 20:
+    if len(dic["optimization"]) < 20:
         axis.xaxis.set_major_locator(MaxNLocator(integer=True))
     else:
         axis.set_xticks(np.linspace(
             0,
-            optimization[-1][0] + 1,
+            len(dic["optimization"]) + 1,
             7,
         ))
     fig.savefig(