Human Factor Evaluation Human Factor Evaluation for Marine Education by for Marine Education by using Neuroscience Toolsusing Neuroscience Tools

N. N. ΝΝikitakosikitakos D. D. PapachristosPapachristos

ProfessorProfessor Ph.d. candidate Ph.d. candidate

Dept. Shipping, Trade and TransportDept. Shipping, Trade and Transport

University of AegeanUniversity of Aegean

MASSEP 2013

May 2013

www.stt.aegean.gr

Human Factors Evaluation Research Methodology Case Study

CONTENTSCONTENTS

www.stt.aegean.gr

HUMAN FACTORS HUMAN FACTORS EVALUATIONEVALUATION

www.stt.aegean.gr

Maritime education user’s satisfaction objective criteria satisfaction phenomena

Human Factor Evaluation Human Factor Evaluation (1)(1)

www.stt.aegean.gr

mixed approach to Human Factor evaluation

ship’s bridge equipment usability and educational evaluation ship bridge interactive systems neuroscience tools of gaze tracking &

speech recording for measuring emotional user responses

Usability testing

Human Factor Evaluation Human Factor Evaluation ((22))

www.stt.aegean.gr

Human Factors EvaluationHuman Factors Evaluation ( (33))Neuroscience toolsNeuroscience tools

www.stt.aegean.gr

Human Factors evaluation ship manipulation systems design (interactive technologies) ergonomic few applications in industry cognitive ergonomics

Human Factor Evaluation Human Factor Evaluation ((44))

www.stt.aegean.gr

Usability has been defined by ISO 9241 as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use”

Human Factor Evaluation Human Factor Evaluation ((55))

www.stt.aegean.gr

Effectiveness means accuracy and completeness with which users achieve specified goals

Efficiency means resources expended in relation to the accuracy and completeness with which users achieve goals

Satisfaction means freedom from discomfort and positive attitudes towards the use on the product

Human Factor Evaluation Human Factor Evaluation ((66))

www.stt.aegean.gr

RESEARCH METHODOLOGYRESEARCH METHODOLOGY

www.stt.aegean.gr

Defining satisfaction concerns the following parameters, which are being investigated: concerning software and educational

scenarios, system usability as far as the system per se

is concerned (total functionality), as well as the individual training and

technical characteristics that complete the teaching act.

Research Methodology Research Methodology ((11))

www.stt.aegean.gr

The research questions (RQ) set by the suggested

research framework as follows: RQ-1: There is a relationship between the

user’s optical attention to the user’s satisfaction either for the software or the scenario?

RQ-2: There exists a relationship between the training characteristics, user satisfactions and by extension optical attention?

Research Methodology Research Methodology ((22))

www.stt.aegean.gr

The suggested research method aims at interpreting, determining and evaluating the data of the biometric tool in combination with the conventional methods (qualitative, quantitative) results based on the factors (relationships) that is possible to influence the user’s satisfaction

Research Methodology Research Methodology ((33))

www.stt.aegean.gr

Research Methodology Research Methodology ((44))

Measurements software

scenario

Quantitative data (questionnaires)

Qualitative data (interview)

Relationships between parameters-factors

Interpretation procedure

www.stt.aegean.gr

CASE STUDYCASE STUDY

www.stt.aegean.gr

The case study aims the following: the evaluation of the user satisfaction

from using the ECDIS software and scenario and

educational evaluation of ECDIS from the user’s point of view (opinions).

Case study Case study ((11))

www.stt.aegean.gr

first (random) sampling (January 2012 until May 2012), in the Information Technologies Laboratory of the National Marine Training Centre of Piraeus

31 Marine officers video recording of ~23 minutes per

student

Case study Case study ((22))

www.stt.aegean.gr

Case study Case study ((33))

ECDIS operation

ECSIS Lab room

Eye tracker

“Face Analysis“ software - station

experiment

www.stt.aegean.gr

Stage 1: Information about the experiment, Presentation of the acceptance document by the user-trainee (estimated time duration 5 - 10 minutes)

Stage 2: Completion of a user’s profile and of the assessment survey concerning educational and technical characteristics (questionnaire, T3) by the trainee (estimated time period 10 - 15 minutes)

Stage 3: Equipment installation (gaze tracking device) and configuring the parameters (T1)

Stage 4: Video recording (T1) in combination with filling in a work sheet (T3) by the researcher (estimated time duration 20-25 minutes)

Stage 5. Completion of the process (device disconnect) through a semi structured interview & questionnaire (T2, T3, T4) with the user (estimated at 5 – 10 minutes)

Case study Case study ((44))

www.stt.aegean.gr

Tool-1 (T1): the optical data registration will be conducted by the “Face Analysis” software in connection with a Web camera set on the computer in which there is the subject of the research (ECDIS)

Tool-2 (T2): Use of a microphone for voice recording (interview)

Tool-3(T3):Questionnaires using for opinion/attitudes/expectation/self-evaluation and observing

Tool-4 (T4): Usability assessment tool SUS

Case study Case study ((55))

www.stt.aegean.gr

Case study Case study ((66))

Eye gaze vector

Schedule of eyes & head pose

Distance from monitor

Head roll (angle), HR

Xo

Eyes Quality parameter (eye gaze trucking) values (horizontal)>0: mean out of screenvalues (vertical)→-18 view of the center of the screen

~0 attention in screen~1 & >1 no attention

>1 close to the screen<1 away from the screen

Values >10o degrees, (high mobility)Values <10o degrees (attention depending on the scenario ELHR= --------- HL

Horizontal Level, HL

Eye Level, EL

Βiometric tool parameters interpretation (‘Face Analysis’)

www.stt.aegean.gr

The data of experiment come from three sources

(by using SPSS, Excel): questionnaires, SUS Tool, optical data (gaze tracking) and Interviews (voice recording)

Case study Case study ((77))

www.stt.aegean.gr

The results are shows: a relationship between Gaze parameter and Usability

assessment of users. The gaze parameter depending from SUS score. It shows attention increases as assessment from ECDIS software (RQ-1),

a relationship between SUS score and training characteristics (total assessment, time schedule) and a strong relationship between ECDIS satisfaction and Scenario Satisfaction (RQ-2). It seems the scenario operation depending from software environment (navigation, interface),

High usability for ECDIS software (questionnaires evaluation, SUS tool) and high score for Training program evaluation (National Marine Training Centre of Piraeus).

Case study Case study ((88))

www.stt.aegean.gr

Case study Case study ((99))Marine Officers Male

(29)Female(2)

Age’s scale

24-35 years 16 (51.6 %) 2 (6.45%)

36-45 years 6 (19.35%) 0

>45 years 7 (22.5%) 0

Officer order

A’ 13 (41.9%) 0

B’ 4 (12.9%) 0

C’ 12 (38.7%) 2(6.45%)

Eye diseases 10 (32.2%) 0

Eye Operation 1 (3.2%) 0

Sample’s structure

www.stt.aegean.gr

Case study Case study ((1010))

Marine Officers Training Program ECDIS

Total Assessment Time Schedule Educational Goals Navigation Interface Multimedia

Evaluation scale

Very high 15 (48.3 %) 18 (58.06%) 19 (61.25%) 15 (51.6 %) 12 (38.7%) 12 (38.7%)

High 7 (22.6%) 7(22.6%) 1 (3.22%) 9(19.35%) 13 (41.9%) 16 (51.6 %)

Medium 6 (19.35%) 4 (12.9%) 2 (6.45%) 7 (19.35%) 6 (19.35%) 3 (9.6%)

Low 3 (9.6%) 2 (6.45%) 0 0 0 0

Total 31 (100%) 31 (100%) 31 (100%) 31 (100%) 31 (100%) 31 (100%)

ECDIS – Training program Evaluation

www.stt.aegean.gr

Case study Case study (1(111))

No Variable correlated Spearman’s rho Sig. (2-tailed) Remark

1 SUS score – ECDIS Satisfaction .453 .011 Positive

2 SUS score - Interface .469 .008 Positive

3 SUS Score - Gaze .393 .029 Positive

4 ECDIS satisfaction – Total Assessment .398 .026 Positive

5 ECDIS satisfaction – Time Schedule .468 .008 Positive

6 ECDIS satisfaction – Navigation .531 .002 Positive

7 ECDIS satisfaction – Interface .563 .001 Positive

8 ECDIS satisfaction – Scenario Satisfaction .708 .000 Positive

9 Scenario Satisfaction - NAvigation .373 .039 Positive

Correlations between variables of research tools

Thank youThank younnik@aegean.grnnik@aegean.gr