Using Open-ended Problem-solving Tests to Identify Students’ Mathematical Creative Thinking Ability

Accepted: 13.02.2021 This study aimed to examine an open-ended problem-solving assessment tool that can be used to measure the profile of students’ mathematical creativity based on the type of mathematical creativity. The open-ended problem-solving assessment consists of open-ended problem-solving sheets and interview guidelines to measure the profile of students' mathematical creativity. One hundred and five students were asked to solve open-ended problems. From each group, one student was selected to be interviewed with the aim of obtaining a more detailed explanation of each of these types. The two types can be explained as follows. The “very creative” students were those who could perform cognitive flexibility and cognitive fluency in solving open-ended problems. The other type of students was creative students who were able to think flexibly (cognitive flexibility) when solving open-ended problems. Cognitive flexibility includes the ability to describe various solutions and provide more than one answer to a problem. Cognitive fluency includes the ability to interpret the answer smoothly and accurately without any constraints. It may also refer to the ability to correct a wrong answer even though there was a mistake during the operating process. The analysis results showed that the open-ended problem-solving assessment was an effective tool to measure students' mathematical creativity in terms of results (products) and processes. In terms of results (products), it was able to determine the type of students’ mathematical creative ability, namely very creative and creative, while in terms of processes, it was able to describe the differences in the cognitive profile of the two types.


Introduction
For over decades, many researchers have been interested in developing an instrument to assess creative thinking ability as well as identify and measure the original ideas (products) of innovative individuals (Silvia et al., 2012). The assessment of creative thinking ability has been focused on creative products, creative cognition, creative features, and creative achievement. In general, Getzels dan Csikkszentmihalyites state that the most common test used to measure creative thinking ability is the problem-solving test, since the ability to discover problems is believed as the primary category to analyze creative processes (Singer & Voica, 2015).
Even though routine problem-solving tests can be used to evaluate cognitive achievement, they cannot provide sufficient measures for assessing mathematical creative thinking ability. Therefore, another type of instrument that is associated with this domain in mathematics should be developed. It is expected that this instrument is able to collect more complete and more meaningful data on the success of learning. Mathematical creative thinking ability in this study refers to one's tendency to discover novelty in solving mathematical problems.
Measuring individuals' creative thinking ability is one of the important features of education.
The tests that are usually used for measuring creative thinking ability are Torrance Tests of Creative Thinking (TTCT) (Bart, Hokanson & Can, 2017). According to Hokanson, the TTCT developed by Torrance and his colleagues Torrance are very popular and available in 35 languages, TTCT primarily aim to identify gifted children (Kim, 2006). The TTCT criteria developed by Torrance have not been applicable to measure mathematical creative thinking ability, especially mathematical communication ability because mathematical creative thinking ability is often considered more specific than creative thinking ability in general (Singer & Voica, 2015).
Numerous researchers proposed psychometric approaches to measuring creative thinking ability of a wider population, such as of famous artists or inventive scientists (Althuizen, Wierenga & Rossiter , 2010;Stenberg & Stenberg, 2012). The psychometric approaches contain tests to measure creative or divergent thinking ability. Based on the explanations, it is clear that creative thinking ability is also known as divergent thinking ability. Creative thinking ability can be evaluated using divergent thinking tests, such as the Guilford's Alternate Uses test, that can predict someone's creative performance (Kaufman, Plucker & Baer, 2008;Runco & Acar, 2012).
Open-ended tests are an effective instrument to measure creative thinking (Kaufman et al., 2008;Kaufman, Kaufman & Lichtenberger , 2011). Someone with divergent thinking is able to produce new and original solutions to open-ended problems. The statement suggests the link between creative thinking and open-ended problems (Dietrich & Kanso, 2010).
The theoretical and practical discussion about creative thinking in the existing references generally addresses psychological and social creativity only. It is rare to find a discussion of creativity in the field of mathematics thereupon many obstacles are found in interpreting it in the field of mathematics learning. This study sought to interpret the ability to think creatively in mathematics, especially at the university level. The theories referred to in this study were collected from references that are applicable for mathematics learning at the higher education level.
Sriraman, (2009) examined the characteristics of mathematicians' creative thinking and found that, in general, the mathematicians' creative thinking process followed the Gestalt's creativity model which consisted of four stages namely preparation, incubation, illumination and verification. Sriraman also described the difference between mathematicians' creative behavior and novice's or learners' creative behavior. The question that may arise when comparing learners' creative thinking and mathematicians' creative thinking is: "what are the limitations of learners' creative thinking?" In fact, novice learners face the difficulty in using words to describe cognitive processes that occur in themselves (Gervás & León, 2014). During the learning process, learners have to develop a coherent mental framework, operators, genres and text types that can assist knowledge integration and transformation processes (Gervás & León, 2014). Furthermore, Gervas & León (2014) revealed that experts tend to have a mental framework that underlies and supports their writing skills while for beginners, the problems must be dealt with the help of general knowledge about how to design artifacts, how to change mental structures and how to solve problems. This shows that the novices' and experts' creative thinking abilities are different; thus, the tools used to measure their creative thinking abilities are also different.
To answer the questions above, this study involved a group of students from a university in Indonesia. Their creative thinking in mathematics was measured using open-ended problemsolving tests (OEPST). Creative thinking in terms of product (result) was evaluated based on the number of distinctive ways proposed by the participants in solving mathematical problems (cognitive flexibility). Meanwhile, creative thinking in terms of process was revealed through in-depth interviews on the reasons why the solutions were proposed.
One of the indicators of mathematical creative thinking ability is cognitive flexibility (Rahayuningsih, Nusantara, As & Susanto, 2019;Singer & Voica, 2015). Cognitive flexibility is fulfilled through cognitive variety (learners propose different problems), cognitive novelty (learners produce new ideas), and change in cognitive framing (learners change their cognitive framework to identify new problems).
The indicators of creative thinking ability have been proposed by some experts. Sriraman (2009), explain that creative learners possess cognitive fluency, cognitive flexibility, and originality. According to Stenberg & Stenberg (2012), elaboration thinking ability also characterizes creative learners. Singer & Voica (2015) employed the organizational theory of creativity in problem posing with cognitive flexibility and revealed that learners who could perform cognitive flexibility were able to pose a new (cognitive novelty) and different (cognitive novelty) solution to a problem. These learners were also able to change their cognitive framework in solving or identifying a new problem (change in cognitive framing). The indicators of creative thinking used in this study were constructed based on the similarities of the creative thinking characteristics presented by the experts above ( Table 1). The characteristics of students' mathematical creative thinking ability in open-ended problemsolving tests (OEPST) are explained in detail as follows: (1) The flexibility cognitive components introduced by (Singer & Voica, 2015) include cognitive variety, cognitive novelty and change in cognitive framing. Learners who possess cognitive flexibility are able to propose varied problems (cognitive variety), generate new ideas to solve problems (cognitive novelty) and change or identify new ways to solve problems (change in cognitive framing). Cognitive variety refers to the diversity of ideas used to solve problems that exist in the same group (Fisher et al., 2012) or the diversity of cognitive paths or perspectives (Furr, 2009). (2) Cognitive novelty refers to the concept related to student learning and mastery of content (Orion & Hofstein, 1994). Furr (2009), adds an external perspective in the form of previous experiences, especially on successful experiences to a phenomenon called cognitive framing. Cognitive framing according to (Jack et al, 2020) can be demonstrated by trying to solve new problems through the use of certain strategies. Based on this description, it appears that cognitive flexibility includes novelty and flexibility.
(3) Cognitive elaboration includes the ability to produce new or unorthodox ideas (novelty).
It is obvious that cognitive elaboration and cognitive novelty have similar components.
The construction table above indicates the omission of cognitive fluency from the indicators of creative thinking ability in the research (Singer et al., 2017). Even so, we still considered this component important in this study. In addition to the diversity of solutions (cognitive flexibility), creative thinking also requires cognitive fluency. The purpose of adding cognitive fluency in this study was to assess the appropriateness of the solutions provided by the students on the tests. Based on these considerations, we decided to include cognitive fluency as an indicator to measure the students' (pre-service teachers') creative thinking in solving mathematical problems.
For this reason, we would incorporate cognitive flexibility and cognitive fluency into the assessment of students' creative thinking ability. Learners were said to be very creative if they were able to solve open-ended problems using cognitive flexibility and cognitive fluency. Learners were said to be creative if they were able to solve open-ended problems using cognitive flexibility alone. Cognitive flexibility includes the ability to describe various solutions and provide more than one answer. Cognitive fluency includes the ability to interpret answers smoothly and precisely without constraints and fix the answers despite making mistakes in the operational process. This study aimed to reveal students' mathematical creative thinking ability in terms of product and process using open-ended problem-solving tests (OPST). Creative thinking in terms of product (result) was evaluated based on the answers given by the participants, while creative thinking in terms of process was seen based on the cognitive processes existing while solving problems using OPST.

Method
This study belonged to a qualitative study. It aimed to obtain information about the use of open-ended problem-solving tests (OEPST) in measuring students' mathematical creative thinking ability. The data of this study were collected using OEPST and interview. According to (Creswell, 2012 ;Fraenkel, et. all, 2011), this study can be classified as an exploratory study with a qualitative approach. The participants of this study were selected based on the following criteria: (1) the participants were registered in the seventh semester by the university; the seventh-semester students were considered to have more spare time for interviews; (2) the participants had completed the Field Experience Program 1 so that it could be ensured that they had experienced teaching mathematics to the elementary school students; (3) the participants were able to communicate ideas effectively (communicative); (4) the participants possessed the ability to solve the problems given. Therefore, based on these criteria, 4 out of 105 students were selected. Two of the four students would represent the creative group and the other two students would represent the highly-creative group. The students were considered very creative if they possessed both cognitive flexibility and cognitive fluency. However, the students who only possessed cognitive flexibility would be assigned to the creative group. The creative thinking ability of the four participants would be examined based on the OEPST product and process. Therefore, the result of the analysis would provide a clear description of students' mathematical creative thinking ability in solving open-ended problems.
The researchers acted as the main instrument in this study. The other instruments used in this study were open-ended problem-solving tests (OEPST) and interview guidelines. The OEPST used in this study were developed from Siswono et al.'s open-ended problem-solving tests ( Siswono et al. 2012) . The differences between the before-and after-development OEPST can be seen in the following table.  (Siswono et al. 2012) 1. Use the seven tangrams above to form a different twodimensional shape. 2. Determine the area of the two-dimensional shape. 3. Use only some of the tangrams above to create another two-dimensional shape. 4. Determine the area of the two-dimensional shape formed from several tangrams. 5. Calculate the area of each known tangram.
Examine the following rectangle, a. Create a two-dimensional shape that has the same area as the rectangle. b. Draw at least two two-dimensional shapes that have the same area as the rectangle. c. Examine one of the two-dimensional shapes in (b) and figure out a different way to draw the shape.
The instrument created by (Muhtarom, Juniati & Siswono., 2019) was originally used to measure the creative thinking ability of elementary students. Since this study involved university students, it was decided to develop Siswono et al.'s instrument by increasing the difficulty level of the problems.
The participants' creative thinking ability was measured based on their OEPST answers that met the criteria of cognitive flexibility and cognitive fluency. The data were supplemented by the results of the interviews and video transcripts. During the interviews, the participants were provided with the opportunity to express their ideas freely so that their creative thinking pattern could be easily identified. The interviews were also conducted to reveal the strategies and methods used by the participants to solve the problems and examine their perceptions regarding the problem-solving process. The results of the analyses were used to outline the cognitive profile of the students' problem-solving process. To draw a conclusion, the data from the interviews were synchronized with the data from the tests. The task that subjects were requested to solve is presented in Table. 2. As can be seen, the task could stimulate student to think creatively . Subjects were asked to verbalize their thinking processes while attempting to solve the task. The interviewer used prompts such as ''why do you think so''; ''can you give other reasons''; ''can you show that on the graphs''.
Trustworthiness was enhanced through (a) ensuring that the collected data are accurate and complete, by way of administering the task in a written form and producing a verbatim transcription of each interview shortly after its recording; (b) validating the process of coding and recoding of the different categories via discussions with several mathematics education specialists. Each researcher analysed the data separately and we then discussed the individual findings jointly to reach agreement

Results
The participants' mathematical creative thinking ability was described based on the analysis of the open-ended problem-solving tests' (OPST) product and process. The tests contained five interrelated and hierarchical questions from simple to complex. An analysis was also conducted to see the differences in the cognitive process between the creative and highlycreative groups.
Based on the tests' results, Participant number 1 (P1) was classified into the highly-creative group. This was indicated by the participant's performance in cognitive flexibility and cognitive fluency. Cognitive flexibility, as explained earlier, includes the ability to elaborate solutions and provide more than one solution to a problem, while cognitive fluency refers to the ability to interpret answers correctly or amend wrong answers despite the errors made during the process. The following sections contain the excerpts from the interview conducted with P1: The interview excerpts above show the participant's ability to create three two-dimensional figures from the seven known tangrams. The participant's answers to the question are presented in Figure 1.

Figure 1. Participant 1's Tangrams
It was proven that Participant number 1 was able to provide more than one varied answer. Similarly, P1 was also able to solve problem number 2 correctly. The result of the interview showing P1's ability to solve the problem can be seen in the following excerpts.

R P1
What is the solution to this problem? See number 4. Suppose that tangram number 4 has an area of 1cm 2 . The ratio of the area of tangram 4 and of tangram 3 is 1 : 2, so are 4 and 5 , 4 and 2. If tangram 4 has an area of 1 cm 2 thus tangram 3 has an area of 2cm 2 . Tangram 5 has an area of 2cm 2 and tangram 2 has an area of 2cm 2 . Tangram 6 and tangram 4 are congruent, hence tangram 6 = 1cm 2 . The total area of all tangrams is 10 cm 2 . The area of tangram 1 is the same as the area of tangram 7, then 10 cm 2 divided by 2 equals 5cm 2 , so that both tangram 1 and 7 have an area of 5cm 2 .
Despite the mistakes made in the simple calculations, P1 showed cognitive fluency in solving the problem. P1 then realized the error and attempted to correct his answer:  Participant Number 1 carried out a thinking process at the highly-creative level. The first thing the participant did was to observe the question. Although it took a long time to decipher his creative ideas, P1 was eventually able to come up with many solutions by comparing the area of each tangram, assuming the smallest to the largest tangram area, finding different settlement patterns, using all or some of the existing tangrams to arrange a new tangram. P1 succeeded in finding the connection between the problem with real life contexts and with experiences that had been encountered before, as revealed in the following interview excerpts:

Elaborate why you provided this answer (pointing at the answer).
It is like a puzzle. I am used to rearranging a puzzle with the base frame; therefore, it is quite challenging for me to arrange a puzzle without the base frame. I have difficulty arranging the tangrams into different shapes. why is that?
it is difficult to find the pattern.
The excerpts above indicate the tendency of P1 to be more flexible in responding to external stimuli. Cognitive flexibility is one's ability to respond to changes in tasks and produce representations and actions that are tailored to change tasks in a particular context (Deak, 2003;Singer & Voica, 2015). Cognitive flexibility refers to the ability to adjust problem-solving strategies to the current situation or modify the strategies used (Sriraman, 2009). From this description, it can be inferred that P1 is capable of using different strategies in solving problems in daily life. P1 has the ability to see problems with different perspectives and modify strategies to solve more complex problems.
Unlike P1, Participant Number 2 (P2) was categorized into the creative group because the participant could only show cognitive flexibility without cognitive fluency. The results of the interview conducted with P2 are presented as follow:

R P2
: : Can you understand the first problem? There are 7 tangrams in the picture. I was asked to put the 7 tangrams together to form a different two-dimensional structure (P2 voluntarily elaborated on the answer). The first two-dimensional shape was a square. I considered tangram with the largest size as tangram number 1 (naming the tangram). Because there are two large tangrams of the same size, so the next tangram is assigned as tangram number 4 (mentioning the name of the two-dimensional shape. Although he was able to distinguish between the seven shapes, he failed to mention the correct name of each shape).

Figure 3. P2's Tangrams
From the interview excerpts above, it is clear that P2 was able to decipher his creative ideas and provide diverse answers. Although he failed to mention the name of each shape, P2 was able to distinguish the seven known tangrams.

right angles of tangram VI because of its tilted position. What I see is an isosceles triangle that is congruent with tangrams II and V …
In solving problem number one, P2 found a pattern to form a new shape from the seven known tangrams. Cognitive flexibility in P2 was seen through his strategy in determining the problemsolving pattern and solving the pattern in a variety of ways (cognitive variety). However, P2 had difficulty answering question number 2, namely determining the area of the shape formed from the seven tangrams. From the description above, it is clear that P2 was able to perform creative thinking at the creative level only because P2 was unable to find the areas of the new tangrams. The difficulty found by P2 in determining the area of the new tangrams was caused by the tendency of P2 to solve mathematical problems procedurally (which does not meet the characteristics of cognitive fluency). Therefore, when faced with incomplete problems, P2 experienced difficulty. The following are the excerpts of the interview: What about problem number 2? I am confused with it. Why? because one of the sides is unknown. Can you elaborate? Before I look for the area of a new tangram formed from the seven known tangrams, I must find the area of each of the seven known tangram, but how do I determine the area while the length of one side of the tangram is unknown? Have you ever encountered a similar problem before? Never.
Participant number 3 (P3) produced a new idea while remembering the test he had done before. His creative idea started with making random patterns and calculating the area by tracing a drawing of tangram on printed paper. The idea was not triggered by the surrounding circumstances. Linking the idea to the context of the open-ended tests gave birth to an arbitrary pattern as a first step to forming a new structure consisting of all known tangram. Tangram that was formed lacks the resemblance of a standard structure but was more likely to form a structure initially imagined resembling unique shapes. The interview excerpts indicated that P3 had reached the implementation stage of new ideas. The ideas were realized in the form of new tangrams resembling crab claws, fish and train cars (Figure 4).

Figure 4. P3's Tangram
Participant number 3 (P3) was able to employ different strategies to find the solution to the problem. The participant was able to perform cognitive flexibility in solving the problem. In addition, the same process was carried out to discover and test the different, demonstrating his fluency cognitive in problem-solving. Therefore, P3 can be categorized as highly-creative. In line with the statement of Ibrahim & Widodo (2020), students who can smoothly or fluently think are students who can express many ideas, answers, and are capable of problem-solving.
On the other hand, Participant number 4 (P4) first calculated the area of each known tangram and then found several answers with different characteristics, namely drawing various shapes that resembled train cars, submarines and kites. The student easily deciphered the answer to the next problem in the same way, but the results were indeed different. This implied that the participant was able to show cognitive flexibility when solving mathematical problems. The excerpts of interview conducted on P4 can be seen as follows: The participant successfully created three different shapes that resembled a kite, train cars, a submarine, and a trapezium. P4 was able to draw unique shapes from the tangrams provided The participant easily summed up the area of each tangram that formed the new tangram. However, the participant was unable to test the truth of the answer obtained. This proved that the participant was unable to perform cognitive fluency. Thus, it can be concluded that the participant can be categorized as creative as he only met the indicators of cognitive flexibility.

Discussions
Based on the results of data analysis, it can be concluded that open-ended problem solving is an effective instrument to measure students' mathematical creativity skills in terms of results (products) and processes. In line with the opinion of Ibrahim & Widodo (2020) learning through an advocacy approach by presenting open-ended problems has given students ample opportunity to think freely, submit opinions or ideas, and ask questions. Explanation in the form of geometric problems by arranging pieces of tangram can improve students' mathematical creativity, especially in learning geometry. In line with the results of research by Khairiree (2015) that the use of tangram in mathematics learning can improve students' creative thinking and understanding in mathematics, especially in learning geometry. Khairiree further revealed that learning using dynamic geometry software The Geometer's Sketchpad (GSP) is able to create a learning environment that is fun, motivating, thought-provoking and challenging.
During the process of finding the initial idea of solving the problem, there were differences between participants P1 and P3 (highly creative) who favored the "guest and check" strategy and participants P2 and P4 (creative) who favored a sudden resolution. The term guess and check in this study refers to a process to look for information by taking action first. Guess and check strategy is a strategy of guessing and checking whether the answer to the problem given is correct or not (Levin, 2008).
Participants P1 and P3 attempted to build ideas by finding the connection between their existing knowledge and the problems presented in the tests. At this stage, P1 and P3 seemed to think hard to construct a hypothesis by rearranging the tangrams. On the other hand, P2 and P4 made use of relevant contexts found on a daily basis to find the solutions to the problems. Therefore, they looked more relaxed during the idea construction process and it took a short time for them to figure out the answer.
Besides, compared to participants P2 and P4, participants P1 and P3 often looked hesitant and desperate to carry out the initial steps that had been built on their imagination, such as connecting problems with prior knowledge obtained from either learning experiences or daily experiences. Concepts and thoughts come from the subconscious self. Subconscious awareness can be used to test a concept, whether it is accepted or rejected (Mosimege & Engagement, 2016). In fact, a person can only remember a small part of his subconscious process depending on how often that person pays more attention to the ideas that arise in his subconsciousness. Ladd & Troop-Gordon, 2003 ), explains that progress is influenced by an unconscious mentality in the form of imagination and intuition.
Despite feeling under pressure during the imagination process, P1 and P3 were actually entering the productive-imaginative thinking stage. According to Ladd & Troop-Gordon (2003), Sirajuddin, et al. (2020, Ikram et al, (2020) ideas emerge through imagination and anticipation. There are two types of imagination, passive imagination such as daydreaming or dreaming and reproductive imagination such as imagination in science. Furthermore, (Ladd & Troop-Gordon, 2003) explains that reproductive imagination is supported by the ability to reshape past experiences. Individuals with such abilities always have the desire to observe everything and realize their ideas. Such imagination can be called creative imagination. Creative imagination usually results in new discoveries in the form of objects, concepts, or models. Ladd & Troop-Gordon (2003), mentions that doubt, courage, attraction, sincerity, temporary surrender, relaxation/rest, writing, confusion, feeling depressed and efforts in reaching the target, can affect the level of productivity of the imagination. The statement proves that the participants' imagination process belonged to the reproductive process of imagination that gave birth to unique or unusual discoveries that remain logical. The following is a diagram of the differences between the two subject groups, namely the creative and high creative subject groups: Figure 6. The Differences between Creative (C) and High Creative Subject (HC)

Conclusions
Based on the discussion above, it is clear that open-ended problem-solving tests (OEPST) was effective in measuring students' mathematical creative thinking ability in terms of products (result) and process. In terms of product (result), the instrument was able to determine the level of students' mathematical creative thinking ability (creative and highlycreative), while in terms of process, the instrument was able to describe students' mathematical creative thinking processes.
The creative thinking process of the highly-creative group occurred when they are able to demonstrate cognitive flexibility and cognitive fluency. The cognitive process consisted of the following stages: 1) Reading all the questions repeatedly; 2) Marking important; 3) Recalling information in the form of concepts; and 4) Doing "guess and check".
One the other hand, the creative group could only perform cognitive flexibility, where they were only able to discover many different solutions but failed to prove them. The cognitive process carried out by the creative group consisted of the following stages: 1) Reading the easy question first; 2) Paying attention to other easy questions and finding the connection between them; 3) Recalling information in the form of daily experiences; and 4) Being spontaneous.
Based on the conclusions, it is strongly recommended that teachers involve learners more in open-ended problem-solving processes to stimulate their creative thinking ability. Teachers can also develop an instrument to measure students' creative thinking process or implement a learning model that can support the development of students' creative thinking ability, especially in mathematics.