For map project from https://qgiscloud.com/Bamse/MapMedievalRealms/ gatthering raw data it's a time consuming task because we need to write manual coordinates from screen to a file.
To automate we can extract coordonates from screen via screenshot, ocr, some processing and writing results to a file.
Thanks to @Beko for initial implementation on linux
Here are a arhive with latest scripts version. ocr-helper-v1.0.zip. Unzip them and put in directory
c:\map\bin
ocr.cmd
. Now is more smarter and ... start talking if something bad happened. To update just replace content of existing script ocr.cmd
with new one from this guide.Fail capture
!Same location
!work.cvs
it say Ok
!So now it's implemented some error detection and you hear them audio to not alt+tab all time to see if coordinates are taken and write in work.cvs
!
So almost no need to alt+tab to do basic tasks.
The plan is to make something similar what @Beko do on linux, but for windows. For this we need to install required software and configure it for our needs.
Assumption :
c:\map
c:\map\bin
. Of course you can install on other place but then you must change scripts and location from this guide.We need:
Note:
All download links and versions are taken as is in 10 december 2022.
For screenshot i use Grenshot
Download link https://github.com/greenshot/greenshot/releases/download/Greenshot-RELEASE-1.2.10.6/Greenshot-INSTALLER-1.2.10.6-RELEASE.exe
This screenshot is great for our scope because can be configured to make just a screen shot of a cropped region of screen and ulterior capturing screenshot can be configured to launch a command (i use this command to trigger script who make ocr and proccess result)
For OCR i use Tesseract for windows
Download link https://digi.bib.uni-mannheim.de/tesseract/tesseract-ocr-w64-setup-v5.2.0.20220712.exe
We need AWK for windows for cleaning results from OCR
Download link https://gnuwin32.sourceforge.net/downlinks/gawk.php
We need nircmd for running silent in background scripts from sheel (cmd) without open window.
Download link https://www.nirsoft.net/utils/nircmd-x64.zip
Optional, But strongly recomend Notepad++ for working with result file and, more important, we can activate tail (live monitoring last lines from a file) for result file to live monitor if coordinates go well in file.
Download link https://github.com/notepad-plus-plus/notepad-plus-plus/releases/download/v8.4.7/npp.8.4.7.Installer.x64.exe
So we install AWK, Tesaract OCR, nircmd in our dedicated directory c:\map\bin
. For other software we do standard instalation in default location. Before start install i make a directory c:\map\kits
and put here all required soft. This look like that now:
For this we do a standard instalation in default location.
Chose language
Accept license
Next
Standard options but be sure to install Open with external command plug-in
. It's essential for our scope!
Option to start with windows automaticaly screenshots program too. Personal i start program manually when i need it. But it's to you!
Start install
Done!
For this we do a standard install but chose to install in directory c:\map\bin
Chose language
Next
Accept license
Next
Next
Here chose to install in directory c:\map\bin\ocr
!
Start install
Done
3.3 AWK for windows
Perform standard install, but chose to install in dir c:\map\bin\awk
Next
Accept license
Chose to install in directory c:\map\bin\awk
!
Next
Next
Next
Start install
Done
For this we don't have a install program, just a arhive. So just unzip it and put the 3 files from archive in directory c:\map\bin
In this moment directory c:\map\bin
will look as this:
Just perform standard install (Next... Next...)
So we have installed all required soft. Now we need a script who take one argument (file name captured from screenshot) and do ocr, process and write results.
Make a file named ocr.cmd
in directory c:\map\bin
with the following content:
REM OCR.CMD
REM Ver 1.0 (10 dec 2022)
REM ----------------------
@echo off
if [%1]==[] goto usage
REM Add to path AWK,Tesseract OCR and bin directory
SET PATH=C:\Map\bin\awk\bin;C:\map\bin\OCR;C:\map\bin;%PATH%
REM Change to work directory
cd c:\map
REM OCR file got as first argument and write result in ocr_out.txt
tesseract %1 "C:\map\ocr_out"
REM Update last cooordinates
copy /Y "current.txt" "last.txt"
REM Process file ocr_out.txt and extract coordinates, round it and write result to current.txt
awk "match($0, /([[:digit:]]+[.][[:digit:]])+.([[:digit:]]+[.][[:digit:]]+).([[:digit:]]+[.][[:digit:]]+)/) { print substr($0, RSTART, RLENGTH)}" ocr_out.txt | awk "{ printf \"\n%%0.0f,%%0.0f,%%0.0f,\", int($1+0.5), int($3+0.5), int($2-1.5)}" > current.txt
REM test if current position is empty (failed to extract coordinates from ocr text)
type nul > blank.txt
fc "blank.txt" "current.txt"
if x%errorlevel%x == x0x goto :failcapture
REM Compare curent coordinates with last one captured, if same ignore
fc "current.txt" "last.txt"
if x%errorlevel%x == x0x goto :samecoordinates
REM Add curent location to work.csv
type current.txt >> work.csv
nircmd speak text "Ok"
goto :eof
:failCapture
Echo "Failed to extract cooordonates from ocr result!"
nircmd speak text "Fail capture"
goto :eof
:samecoordinates
echo "Same location!"
nircmd speak text "Same location"
goto :eof
:usage
@echo Please specify filename of cropped image with coordinates!
@echo Example:
@echo %0 capture.tiff
exit /B 1
:eof
What do this script?
Note:
Coordinates are rounded. First is added 0.5 to x and y. For z i deduct 1.5 because i want height of terrain.
For this prepare a cropped file with cooordinates cropped from game and copy to folder c:\map
Like this for example!
You can download one from here test_capture.tiff and copy to folder c:\map
Open a cmd window and change directory c:\map, set path and execute ocr.cmd to test it.
cd c:\map
SET PATH=C:\Map\bin\awk\bin;C:\map\bin\OCR;C:\map\bin;%PATH%
ocr
ocr test_capture.tiff
You must see something like this:
If all is god 2 files will be create in directory c:\map
ocr_out.txt
that contain last extracted text from imagework.cvs
that contain lines of coordinates processed by awkIn my case at test ocr_out.txt
contain
4310.4 125.9 454.0
and work.csv
contain
4310, 454, 124,
So ocr script perform well!
Next run on same cmd window again ocr script but with nircmd
to test too.
nircmd exec2 hide "c:\map" "c:\map\bin\ocr.cmd" test_capture.tiff
Update! From version 1.0 of
ocr.cmd
will not add 2 lines, just one and for second one script talkingSame location
! and ignore it.
Now if all worked in file work.csv
must find 2 lines:
4310, 454, 124,
4310, 454, 124,
Now we have and tested script who perform OCR. We must trigger this script after we take cropped screenshot with greenshot. For that greenshot must be configured.
If remember on install of greenshot we chose to install plugin Open with external command plug-in
With that we can configure a external command who will executed after screenshot is taken. We need to configure this pluggin to launch nircmd
who will execute silent OCR script.
Open Configure external commands
from tray menu of greenshot (right clik on icon from tray bar)
The external comand config windows. Chose New
Create a new external action named ocr
at executable path chose nircmd
from directory c:\map\bin
and at argument line write
exec2 hide "c:\map" "c:\map\bin\ocr.cmd" "{0}"
and chose OK
to save new external program configuration
Chose Ok
again to close configuration window
After we configured external program, we need to configure grenshot options.
Open preference settings of greenshot (right click on icon from tray bar) and on menu chose Preferences
On tab General
we configure keyboard shortcut to perform screenshots. We care about Capture region
and Capture last region
. I chose to use F6 for capture region and F2 for capture last region. Be free to chose any desired combination here! But be aware if change keys some scripts must be changed too! In particular keystroke for capture last region F2 in this case is used in auto capture scripts and ocr-helper. So if want to run all without modification just use same key as mine!
On tab Capture
we set option more for quality of life, nothing critical, but i recommend to disable Show notification
! Rest is up to you.
On tab Output
we set to put capture image in folder c:\map
with name capture
and image format as tiff
. These are important settings!
On tab Destination
we chose what to do with image after screenshot. We want to save a copy of image in directory c:\map
and to launch external program ocr
wich we configured earlier.
For rest of tabs leave default value, nothing critical for our scope.
For test we can make screnshot of a region of screen what contain some text and check if ocr was executed and text extracted.
So press keyboard shortcut for screnshot a region ( F6 on my setup), set a region with some text.
I select this region:
and after that i check in directory c:\map
if i have cropped image at what i selected and in file c:\map\ocr_out.txt
if text was extracted from image.
In this case i found in file:
For test we can make screnshot of a region of screen what contain some text and check if ocr was
executed and text extracted.
All worked as expected! So all instalation and configuration is done.
What we done so far?
Now how to use for Rising World
coordinates extract?
In short open game, press one time key for Capture region
mark the region to frame the coordinates. Check if result arrived in work.cvs
in directory c:\map
. If all is ok for next just press key to perform Capture last region
and rinse and repeat for what coordinates want to collect!
The long one ...
In work...
In this section i will show to create more scripts to help more on collecting data.
The plan is to make a script who will start autocapture coordinates at regular interval without hit explicit keystroke for capture region.
To implement this one ideea is to make a script with a infinite loop who send on regular interval keystroke who start capture region. But it's nice to can have enable/disable auto capture, so i will create another script who will trigger at a keystroke who will mark if auto capture is enabled/disabled.
But with all number of increased scripts of added functionality, i need a new application to control all of them and trigger proper.
So this is how is born OCR Helper
application! This will take care to start everthing, install keyboard hooks and launch scripts when need. This application is itself a script write on AutoIt. So we need to insstall this too.
This scripts will use nircmd
to implement some functionality, so to not handle global path is better to install nircmd
globally. For this just double click on nircmd from folder 'c:\map\bin' and when dialog is open chose Copy to windows Directory
Accept to copy
WARNINNG: If error is throw with message can't copy then you must run nircmd as Administrator
. The simplest way is to right click on nircmd
from folder c:\map\bin
and chose run as administrator
Now we can run nircmd
from any shell window.
Download kit for full install from AutoIt site.
Direct download link https://www.autoitscript.com/cgi-bin/getfile.pl?autoit3/autoit-v3-setup.zip
Unzip downloaded arhive and start setup.
Next
Accept license
I chose native 64 bit defualt
I chose to run when double click on au3
files
I chose full instalation
I will install in directory c:\map\bin\AutoIt3
to have all programs in same place. But can be installed in default path without problem. Nothing hardcoded to this path.
Hit Install
to start install.
Finish
We need to create them in directory c:\map\bin
. Alternative is to download from here ocr-helper-v1.0.zip a zip with all scripts, unzip and put them in c:\map\bin
Scripts:
Create file ocr-helper.au3
with following content in directory c:\map\bin
; OCR Helper
; Version V1.0 (11 dec 2022)
; Author Bamse
;--------------------------------------------
#include <MsgBoxConstants.au3>
#include <date.au3>
;Prevent running multiple instances of thiss program
$g_szVersion = "OCR Helper V1.0"
If WinExists($g_szVersion) Then
MsgBox(0,$g_szVersion,"OCR Helper already running!")
Exit
EndIf
AutoItWinSetTitle($g_szVersion)
; Set key combination to hook our functions
; F1
HotKeySet("{f1}","Details") ; Show a input box and append text to work.cvs
; F11
HotKeySet("{f11}","ToggleAutocapture") ; Enable/disable autocapture
; F10
HotKeySet("{f10}","StartNewWork") ; Create new empty work.csv
; F9
HotKeySet("{f9}","ExitOCRHelper") ; Exit program
MsgBox($MB_SYSTEMMODAL, "", $g_szVersion & @CRLF & @CRLF & _
"F1 - Enter details about last point added to work.csv" & @CRLF & _
"F11 - enable/disable autocapture" & @CRLF & _
"F10 - Start new work.cvs (current is saved as NNNN_work.csv"& @CRLF & _
"F9 - Quit OCR Helper" )
; Check and if not running start programs
StartGreenShot()
StartAutoCaptureScript()
; infinite loop
While 1
Sleep(100)
WEnd
Exit
; this function call script to enable/disable auto capture, bind default to F11
Func ToggleAutocapture()
;MsgBox($MB_SYSTEMMODAL, "", "enable/disable autocapture")
run("nircmd exec2 hide ""c:\map\bin"" ""c:\map\bin\enable-disable-auto-capture.cmd"" ")
EndFunc
; This close this program, bind default to F9
Func ExitOCRHelper()
$t = MsgBox (4, $g_szVersion ,"Do you want quit?")
If $t = 6 Then
StopAutoCaptureScript()
Exit
EndIf
EndFunc
; this confirm get confirm from user and after call start-new-work.cmd, bind default to F10
Func StartNewWork()
$t = MsgBox (4, "Start New Work" ,"Do you want to start new work?"& @CRLF &"(current work.csv will be is saved as NNNN_work.csv)")
If $t = 6 Then
; pressed YES
run("nircmd exec2 hide ""c:\map\bin"" ""c:\map\bin\start-new-work.cmd"" ")
ElseIf $t = 7 Then
; pressed NO - nothing todo.
EndIf
EndFunc
; This function show a input box, and text entered is append to file work.cvs, bind defualt to F1
Func Details()
; Places the input box in the top left corner displaying the characters as they
; are typed.
Local $sAnswer = InputBox("Details", "Specify more details about last coordinate point colected:", "", "", _
600,150, 600,400)
_WriteLineToFile("c:\\map\work.csv",$sAnswer)
EndFunc
; This function write a text to a file
Func _WriteLineToFile($sFileName,$sText)
local $sWriteOption
If FileExists($sFileName) Then
$sWriteOption = $FO_APPEND
Else
$sWriteOption = $FO_OVERWRITE
EndIf
Local $hFilehandle = FileOpen($sFileName, $sWriteOption)
FileWrite($hFilehandle, $sText)
FileClose($hFilehandle)
EndFunc
; This function check if a window with title "OCR - Auto Capture" exists and if not, start script start-auto-capture.cmd if user confirm.
Func StartAutoCaptureScript()
; Retrieve a list of window handles.
Local $aList = WinList("OCR - Auto Capture")
; Loop through the array displaying only visable windows with a title.
local $found=0
For $i = 1 To $aList[0][0]
If $aList[$i][0] <> "" Then ;And BitAND(WinGetState($aList[$i][1]), 2) Then
; windows exist
;MsgBox($MB_SYSTEMMODAL, "", "Title: " & $aList[$i][0] & @CRLF & "Handle: " & $aList[$i][1])
$found=1
else
EndIf
Next
If $found==0 Then
$t = MsgBox (4, $g_szVersion ,"Script for auto capture is NOT running!"&@CRLF &"Do you want to start it?")
If $t = 6 Then
run("nircmd exec show ""c:\map\bin\start-auto-capture.cmd""");
EndIf
EndIf
EndFunc
; This function check if a window with title "OCR - Auto Capture" exists stop script start-auto-capture.cmd if user confirm.
Func StopAutoCaptureScript()
; Retrieve a list of window handles.
Local $aList = WinList("OCR - Auto Capture")
; Loop through the array displaying only visable windows with a title.
For $i = 1 To $aList[0][0]
If $aList[$i][0] <> "" Then ;And BitAND(WinGetState($aList[$i][1]), 2) Then
; windows exist
;MsgBox($MB_SYSTEMMODAL, "", "Title: " & $aList[$i][0] & @CRLF & "Handle: " & $aList[$i][1])
$t = MsgBox (4, $g_szVersion ,"Script for auto capture is RUNNING!"&@CRLF &"Do you want to stop it?")
If $t = 6 Then
WinClose("OCR - Auto Capture")
EndIf
EndIf
Next
EndFunc
;This function if greenshot is running, and start it if user chose this
Func StartGreenShot()
If Not ProcessExists("Greenshot.exe") Then
$t = MsgBox (4, $g_szVersion ,"GreenShot.exe is not running! This is need for screen capture."&@CRLF &"Do you want to start it?")
If $t = 6 Then
Run("""C:\Program Files\Greenshot\Greenshot.exe""")
EndIf
Else
;GreenShot is running
;MsgBox($MB_SYSTEMMODAL, "OCR Helper", "GreenShot.exe is running!")
EndIf
EndFunc
Create file start-auto-capture.cmd
with following content in directory c:\map\bin
@echo off
REM start-auto-capture.cmd
REM Version 1.0 (11 dec 2022)
REM ---------------------------
REM Set title of cmd window.
REM Don't change this! Is used from another scripts to test if this script is running.
title OCR - Auto Capture
REM start infinite loop
echo Start infinite loop... To stop auto capture use CTRL+C or close this window!
echo You can enable/disable with keystroke F11 (default one if not changed!)
REM Set paths
SET PATH=C:\Map\bin\awk\bin;C:\map\bin\OCR;C:\map\bin;%PATH%
c:
cd c:\map\bin
IF EXIST c:\map\bin\autocapture.enabled (
nircmd speak text "Auto capture is enabled"
) ELSE (
nircmd speak text "Auto capture is disabled"
)
:start
REM sleep 3000 ms, autocapture every 3 seconds.
nircmd wait 3000
REM Send keystroke for region capture. This is configured for F2 if followed my guide.
REM If you configure another key for capture region must be modified with your keys!
IF EXIST c:\map\bin\autocapture.enabled (
REM Autocapture is enabled trigger it
nircmd sendkey f2 press
) ELSE (
REM Autocapture is disabled. Nothing to do
)
goto start
Create file start-new-work.cmd
with following content in directory c:\map\bin
REM start-new-work.cmd
REM Version 1.0 (11 dec 2022)
REM -----------------------------
@Echo off
setlocal enabledelayedexpansion
REM Set paths
SET PATH=C:\Map\bin\awk\bin;C:\map\bin\OCR;C:\map\bin;%PATH%
rem Chnage on work directory
c:
cd c:\map
rem search last backup file for work.cvs
dir /b *_work.csv | sort /r > out.tmp
set /p fileName=<out.tmp
del out.tmp
rem set default value if is no backup file created
if [%fileName%]==[] set fileName=0000
rem keep only left 4 chars
set numberPart=%fileName:~0,4%
rem extract number and increment
set /a num=10000%numberPart% %% 10000
set /a num=num+1
set numberToSpeach=!num!
rem Pad with 0 on left and make new filename
set "NPAD=0000000000!num!" & set "NPAD=!NPAD:~-4!"
set backupFileName=!NPAD!_work.csv
echo New backup file is !backupFileName!
rem Copy backup
copy work.csv !backupFileName!
rem Create new empty work file.
echo | set /p dummyName=x,y,z,Details > work.csv
nircmd speak text "Backup file is !numberToSpeach!"
Create file enable-disable-auto-capture.cmd
with following content in directory c:\map\bin
REM enable-disable-auto-capture.cmd
REM Version 1.0 (11 dec 2022)
REM ---------------------------
REM Enable/disable autocapture
@echo off
REM Set paths
SET PATH=C:\Map\bin\awk\bin;C:\map\bin\OCR;C:\map\bin;%PATH%
c:
cd c:\map\bin
IF EXIST c:\map\bin\autocapture.enabled (
del c:\map\bin\autocapture.enabled
type nul > c:\map\bin\autocapture.disabled
echo "Auto capture is disabled"
nircmd speak text "Auto capture is disabled"
) ELSE (
del c:\map\bin\autocapture.disabled
type nul > c:\map\bin\autocapture.enabled
echo "Auto capture is enabled"
nircmd speak text "Auto capture is enabled"
)
So we have now scripts in directory c:\map\bin
. All scripts are launched via main script OCR-Helper
. This is a AutoIt
script. Can be launched with double click, but for easy launch better we convert it in standard executable and create normal shortcut fo easy launch!
Build standard executable from AutoIt script ocr-helper.au3
Navigate in directory c:\map\bin
from Explorer, Right click on ocr-helper
and from meniu chose
As you see a executable called ocr-helper.exe is created from script.
For convenience i recommend to make a shortcut on dektop to easy launch!
This will finish installation and configuration!
What do this scripts from c:\map\bin
?
ocr.cmd - Perform OCR on captured cropped screenshot
This is launched automatically after a region screnshot is made with greenshot. He do OCR , write results in file c:\map\ocr_out.txt, clean data with awk and add data collected to work.csv
He detect if ocr failed and say Capture failed
. If try to add same location as last added it say Same location
and ignore data. If all ok he say Ok
ocr-helper.au3/ocr-helper.exe - OCR Helper control scripts
This main script is controller for all other. At launch check if GrenShot and auto capture script are running and if no, offer option to start them automatically.
After that he install keyboard hook bound to our scripts and start on infinite loop on waiting of keystroke. If some keystroke is pressed by user he will execute corresponding action directly or via aditional scripts from C:\map\bin
. To quit use default keystroke F9.
start-auto-capture.cmd - Script who do auto capture
This script is responsable for auto capture screenshots at regular interval (3 sec). This is done by injecting keystroke F2 (default assignation if followed this guide). This script must be running iff want auto capture to happen. It can be started manually or by OCR-helper automatically.
This script is triggered by key default F10 via ocr-helper
and backup current work.csv
and create a new one. For backup he search last number of file NNNN_work.csv
and increment by 1
This script toggle enable/disable auto capture flag who is check by auto-capture
script. So even if background script for auto capture is running, he will capture only if flagg for auto capturing is enabled.
More than these scripts, ocr-helper
internally will do one more function: Can add some details in work.csv
about last point of coordinates colected via keystroke F1
After all instalation annd configuration we have system ready for collect data.
Step by step how to use:
Start ocr-helper script/executable (the easy is from shortcut from desktop to executable created from script).
At start a help with current keystroke are displayed.
If greenshot is not running, you can start if chose Yes
NOTE:
GreenShot is launched from default instalation pathC:\Program Files\Greenshot\Greenshot.exe
.
If you installed in other location must change path inocr-helper.au3
and make executable again!
If auto capture script is not running, you can start if choose Yes
If you start, script windows will open and running autocapture in background
Now all is ready! In this momment you have, if followed guide and not change any key:
ocr-helper
. This uninstall keyboard hooks.work.csv
. The old one is saved as NNNN_work.csv
Next is to open game ! After that, in game, press 2 times F3 to display current coordinates from game. At this moment you must use once F6 to define a frame box arround coordinates.
Optional, but strongly recommnded is to open file c:\map\work.csv
with notepad++
and for this make a window size desired and activate option from meniu View
-> Monitoring (tail -f)
with this option notepad++ will monitor changes from files and refresh automatically. Another option we must set is from same menu View
and chose Allways on top
or Post-it
to keep windows on top even in game (game is set to run on full screen window).
Next pres F2 and check if current coordinates are displayed on no notepad on file work.csv
If all ok, you are ready to start a new collecting data. Begin with F10 who will create a new empy work.csv
Chose yes and you well see in notepad++ a empty file:
Press F2 and colect first point manualy. You see in notepad++ a new line with coordinates. After that press F1 to open dialog to add some details about last point colected as you see in image bellow
As you see in work.csv arrive text entered in dialog!
The next step is to collect more data manually with F2 or to activate auto capture with F11. On auto capture just walk and in work.csv data is collected at some intervals. Arrived at some point who want to add details and are on auto capture? No problem or just disable and switch on manual mode, ar just stay in same place and scripts don't add new lines of data, just say Same location!
. Pres F1, enter desired details and continue, manual or auto capture! Soon you coolect all data with minimal efort!
After all work of cartographer is done you can press F9 to quit ocr-helper
.
Chose Yes
to close
If background script for auto capture still running you can close too
In this moment still work manual capture via F2 because greenshot is still running (if not manual closed) but all advanced feature like auto capture or input of details for last coordinates is not available. To have all again just launch again OCR-Helper